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Abstract 


In  October  1995,  a  CMOS  brain  chip  consisting  of  two  8x17  multiplexed  sub-arrays 
designed  to  measure  electrical  potentials  at  the  cortical  column  level,  was  implanted  on  the 
somatosensory  cortex  of  a  laboratory  rhesus  monkey.  Electroencephalograph  (EEG)  and 
averaged  evoked  response  (AEG)  data  were  taken  over  a  period  of  40  minutes.  The  brain 
chip  was  replaced  with  an  identical  chip,  and  data  were  again  taken  for  40  minutes.  In 
both  instances  AEG  signals  of  approximately  150  /jVpp  were  recorded.  Additionally,  the 
first  implanted  chip  recorded  three  phases  of  data:  1)  AEG;  2)  large  clock  noise  (during  a 
period  where  the  chip  appears  to  have  burned  the  cortex);  3)  AEG-like  signals  of  magnitude, 
«  100  liVvv,  with  substantially  improved  signal  to  noise  ratio. 

All  data  were  taken  while  the  monkey  was  under  general  anesthesia.  The  monkey  was 
euthanized  immediately  after  the  experiments,  due  to  a  pre-existing  abdominal  cancer. 
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1.  Introduction 


1.1  Justification 

The  creation  of  highly  autonomous  and  reliable  computer  systems  for  weapons  control  and 
information  processing  may  require  new  knowledge  of  the  human  brain.  Man  has  created 
digital  engines  capable  of  performing  highly  repetitive  and,  in  some  cases,  extremely  complex 
tasks  in  a  fraction  of  the  time  required  for  a  human.  Yet,  the  fundamental  limitation  of  these 
machines  is  that  all  possible  variation  must  be  predicted  by  the  programmer,  so  that  the 
computing  system  has  instructions  for  handling  all  situations.  The  challenge  is  to  build  a 
system  wherein  the  programmer  does  not  have  to  foresee  all  outcomes,  and  the  computer  can 
handle  novel  situations. 

The  field  of  pattern  recognition,  determining  the  presence  of  a  particular  signal  based  on 
patterns  in  information,  has  many  of  these  same  challenges  and  would  benefit  greatly  from 
advances  in  ‘intelligent5  processing.  An  example,  pertinent  to  the  intelligence  community  is 
text  reading.  Considering  an  automated  text  reader  with  an  accuracy  rate  of  99%  (reasonable 
for  a  given  set  of  text  with  a  specific  amount  of  distortion-such  as  facsimile  distortion) ,  and 
the  fact  that  each  page  of  a  document  can  easily  contain  more  than  1,000  characters,  we 
would  expect  at  least  10  character  recognition  errors  per  page.  By  way  of  comparison,  a 
human  will  typically  avoid  all  character  errors.  Accuracy,  therefore,  dictates  that  a  human 
proof  read  the  results.  This  being  the  case,  it  is  often  advantageous  to  simply  use  humans  to 
type  in  the  data  without  machine  assistance. 

This  limitation  is  seen  in  the  seemingly  simple  task  of  reading  handwritten  zip  codes. 
Figure  1.1  shows  a  digitized  image  of  a  poorly  written  numeral  ‘05  that  is  consistently  mis¬ 
taken  as  a  numeral  ‘2’  by  many  pattern  recognition  systems  considered  to  have  reasonable 
performance.  However,  it  does  not  look  like  a  ‘2, 5  implying  that  humans  use  different  fea¬ 
tures  to  recognize  numerals  than  the  machines  in  question.  It  should  be  obvious  that  this 
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error  could  be  corrected  by  using  different  features,  possibly  Fourier  coefficients,  instead  of 
those  used  in  the  example  systems.  However,  changing  the  features,  or  even  simply  adding 
new  features,  typically  causes  different  errors.  Given  the  problems  exhibited  by  man-made 
machines,  we  must  ask,  “How  can  people  know  that  Figure  1.1  is  a  ‘0’?” 
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Figure  1.1:  Handwritten  numeral  ‘0’  that  is  mistaken  for  a  numeral  ‘2’  by  pattern  recognition 
systems 

Whether  due  to  its  chosen  feature  space,  preprocessing  algorithms,  classification  al¬ 
gorithm,  or  vastly  parallel  structure,  the  human  brain  is  the  premiere  pattern  recognition 
machine.  Humans  can  recognize  familiar  patterns  in  a  wide  range  of  situations,  including 
time  distortions,  position  changes,  rotation  and  scale  variances,  and  others. 

Beyond  simple  pattern  recognition,  a  human’s  ability  to  take  novel  input  and  create 
associations  based  on  that  input  in  order  to  keep  a  running  model  of  its  environment,  makes 
it  the  most  powerful  decision  making  device  known. 

Information  about  how  the  brain  takes  raw  sensory  information  and  processes  it,  may 
translate  to  great  advances  for: 

•  Intelligence-image  analysis,  character  reading,  and  message  detection 

•  Weapons  development-autonomous  cruise  missiles,  and  local  detection  of  enemy  forces 

•  Health-cancer  detection,  and  other  laboratory  tests 

•  Military  maneuvers-weather  forecasting,  battle  simulation,  and  chemical/biological  agent 
detection. 
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1.2  Problem  Statement  and  Scope 


The  Air  Force  must  be  able  to  record  how  the  human  brain  works  at  the  proper  level  of 
processing  to  begin  to  model  the  genius  that  lies  within. 

The  efforts  supporting  this  thesis  are  directed  at  the  ultimate  goal  of  improving  existing 
pattern  recognition  systems,  by  gaining  new  knowledge  of  the  brain’s  activity.  Toward  this 
goal,  AFIT  is  working  to  organize  current  biological  information  and  existing  neural  network 
simulation  algorithms  into  successively  more  accurate  models  of  human  mental  function.  By 
utilizing  past  brain  chip  work  (see  chapter  3,  page  12)  to  bring  AFIT’s  information  gathering 
capabilities  directly  to  the  cortical  surface,  this  thesis  represents  a  significant  contribution. 
The  specific  goals  of  this  thesis  are: 

1.  Test  and  verify  the  performance  of  newly  developed  brain  chips  prior  to  preparation 
for  implantation. 

•  Voltage  gain  from  the  sensor  pads  to  the  output. 

•  Leakage  noise  from  one  sensor  pad  to  its  neighbors. 

•  Operational  range  of  voltage  measurement. 

2.  Successfully  implant  the  brain  chip  onto  the  sensory  cortex  of  a  test  animal. 

•  Passivation-to  prevent  damaging  interactions  between  brain  and  brain  chip. 

•  Packaging  of  the  implantable  device. 

•  Implantation. 

3.  Verify  the  performance  and  utility  of  the  brain  chip  by  taking  averaged  evoked  response 
data 

4.  Run  apparent  motion  experiments  on  the  test  subject  and  analyze  the  recorded  results 

1.3  Order  of  Presentation 

Chapter  2,  page  4,  provides  a  brief  review  of  the  physiology  of  the  brain.  Chapter  3,  page  12, 
describes  the  development,  design  and  preparation  of  the  AFIT  multi-electrode  brain  chip. 
Chapter  4,  page  26,  describes  the  preparation  for  implantation  and  tests  run  on  the  implanted 
test  animal.  Chapter  5,  page  31,  discusses  the  results  and  makes  recommendations  for  future 
efforts. 
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2.  Human  Brain:  Basic  Form  and  Function 


The  human  brain  is  not  a  monolithic  black-box  for  information  processing.  It  has  definite 
substructure  and  specialized  functionality,  indicated  by  a  wealth  of  research  and  the  hands-on 
experiences  of  brain  surgery.  Figure  2.1  shows  a  detailed  breakdown  of  brain  components. 
This  section  will  describe  a  number  of  these  components  and  provide  a  description  of  the 
functions  attributed  to  each. 
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Figure  2.1:  Human  Brain  [13] 

2,1  Physiology  of  Selected  Structures 

There  are  many  divisions  and  subdivisions  within  the  human  brain.  A  complete  discussion 
is  beyond  the  scope  of  this  thesis.  The  reader  is  encouraged  to  read  more  [8].  Therefore,  this 
section  is  restricted  to  the  cerebellum,  thalamus,  hypothalamus,  amygdala,  hippocampus, 
and  cerebral  cortex. 
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2.1.1  Cerebellum . 

The  cerebellum  sits  at  the  back  of  the  brain  as  an  appendage  of  the  rhombencephalon, 
or  hindbrain.  According  to  Kapit[8],  the  cerebellum  is  the  major  structure  for  motor  co¬ 
ordination.  It  acts  as  a  big  comparator,  checking  the  activity  of  all  voluntary  muscles  against 
the  commands  sent  to  them  and  offering  suggested  corrections.  The  cerebellum’s  position 
between  the  forebrain  and  the  spinal  cord  allows  it  to  perform  its  tasks  rapidly,  providing 
grace  and  precision  of  movement. 

2.1.2  Thalamus. 

The  thalamus  is  the  crossroads  of  the  brain.  Centrally  located,  it -has  connections  to 
most,  if  not  all,  areas  of  the  cortex  [15,  17].  The  thalamus  has  a  number  of  subdivisions  that 
correspond  to  different  structures,  mainly  subdivisions  of  the  cerebral  cortex  [15]. 

The  case  of  Karen  Ann  Quinlan,  who  had  the  misfortune  of  being  in  a  persistent  vegetative 
state  for  10  years,  has  given  new  insight  into  the  role  of  the  thalamus.  Dr.  Kinney  and 
associates  report  that,  though  Ms.  Quinlan  experienced  waking  and  sleeping  states,  marked 
by  increased  eye  movement  and  brain  activity,  she  never  achieved  consciousness  [9],  Autopsy 
findings  revealed  relatively  little  damage  in  the  cortex  but  profound  damage  in  the  sub-cortical 
thalamus  [9].  Thus,  the  conclusion  that  the  thalamus  appears  to  play  a  substantial  role  in 
consciousness  follows. 

Dr.  Ojemann  has  found  that  the  thalamus  has  connections  to  most,  if  not  all  cortical 
areas,  particularly  those  involved  in  language  skills  [18].  In  most  studies  he  was  able  to 
elicit  the  same  language  impairments  by  electrically  stimulating  the  thalamus,  as  he  could  by 
stimulating  the  cortex.  Furthermore,  some  experiments  indicate  that  the  thalamus  was  also 
acting  to  focus  attention  on  particular  information  [18]. 

Again,  most,  if  not  all,  portions  of  cerebral  cortex  have  connections  from  as  well  as 
to  the  thalamus  [12,  15,  16].  These  connections  establish  what  is  called  thalamo-cortical 
loops.  These  loops  are  the  mechanism  for  information  processing  (where  areas  of  cortex  keep 
their  current  hypothesis  of  what’s  happening),  memory  storage,  and  possibly  the  advent  of 
consciousness  itself  [12,  15,  16]. 

Dr.  Mumford  explains  the  basic  architecture  for  thalamus  functions,  particularly  in  ref¬ 
erence  to  the  thalamo-cortical  loops  [15,  16]. 

•  Cortical  areas  learn  recurring  patterns. 
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•  Cortical  areas  continually  try  to  analyze  the  present  situation  in  terms  of  these  patterns, 
generating  hypotheses  that  often  conflict. 

•  These  hypotheses  are  sent  to  the  thalamus  where  a  kind  of  voting  takes  place. 

•  The  consensus  is  broadcast  back  to  the  cortex  as  an  updated  view  of  that  aspect  of  the 
world  dealt  with  by  that  area  of  cortex  [16]. 

2.1.3  Hypothalamus . 

The  hypothalamus  is  a  protrusion  of  the  thalamus  and  has  been  referred  to  as  “the 
head  ganglion  of  the  sympathetic  nervous  system”  by  Dr.  Charles  Sherrington  [8].  Table  2.1 
describes  the  functions  attributed  to  the  hypothalamus. 


Table  2.1:  Purported  Functions  of  the  Hypothalamus  [8] 


Function 

Description  and  Source 

Sympathetic  Responses 

Areas  control  fight  or  flight  response. 
Investigated  by  electrical  stimulation. 

Sexual  Behavior 

Stimulating  or  destroying  sections 
effects  regulation  of  sex  hormones. 

Diurnal  Rhythms 

Areas  respond  to  light  stimuli  and  help 
with  sleeping  and  waking. 

Feeding  Behavior 

Stimulation  causes  increased  appetite. 

Drinking  Behavior 

Stimulation  or  acetylcholine  injection 
induces  drinking.  Destruction  causes 
problems  with  water  retention  and 
imbalances  in  electrolytes. 

Body  Temperature  Regulation 

This  appears  to  be  the  center  of  body 
temperature  regulation. 

Hormonal  Regulation 

Parts  act  like  an  endocrine  gland. 

2.1.4  Amygdala  and  Hippocampus. 

Located  beneath  the  olfactory  cortex,  the  amygdala  and  hippocampus  extend  into  both 
hemispheres  of  the  brain.  Patient  experience  following  brain  surgery  indicates  that  these 
structures  are  crucial  to  memory  formation  [14]. 

Following  surgery  to  control  epileptic  seizures,  every  human  patient  with  post-operative 
retrograde  amnesia  (the  inability  to  form  new  memories)  had  suffered  incidental  damage  to 
the  hippocampus  during  the  surgical  procedure  [14]. 

Animal  studies,  performed  by  Dr.  Mishkin,  Chief  of  the  Laboratory  of  Neuropsychology 
at  the  National  Institute  of  Mental  Health,  have  provided  further  insight  into  the  functions  of 
the  amygdala  and  hippocampus  [14].  In  preliminary  studies  it  was  found  that  the  removal  of 
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only  one  of  the  structures,  be  it  the  amygdala  or  hippocampus,  had  little  effect  on  memory 
processes.  “Animals  whose  amygdala  alone  had  been  removed  were  slow  to  learn  the  associ¬ 
ation  of  stimulus  and  reward,  but  they  were  still  able  to  do  so”  [14].  In  order  to  dramatically 
increase  impairment  and  produce  full  retrograde  amnesia  in  a  macaque  monkey,  both  struc¬ 
tures  had  to  be  damaged  [14].  In  contrast,  just  damaging  a  human  hippocampus  alone  can 
produce  full  retrograde  amnesia  [14]. 

2.1.5  Cerebral  Cortex. 

Unique  to  mammals,  the  cerebral  cortex  is  usually  viewed  as  the  home  of  higher  thought 
processes.  The  human  cortex  is  the  place  where  higher  thought  processes  occur:  sensory 
information  is  broken  apart  and  analyzed,  problems  are  solved,  and  language  is  processed. 

Table  2.2  shows  the  highest  division  and  specialization  of  the  cortex,  the  left  vs.  right 
hemisphere.  Dr.  Kapit  reports  that  97  percent  of  all  humans  have  a  dominant  left  hemisphere 
for  speech  [8] . 


Table  2.2:  Hemispheric  Specializations  of  Human  Cerebral  Cortex  [8] 


Left 

Right 

Verbal 

Motor  Dominant 
Logical 

Analytical 

Linear 

Temporal 

Representational 

Emotional 

Humorous 

Holistic 

Visual/Spatial 

Musical 

Though  there  are  many  other  detailed  processes  of  the  cortex  the  discussion  will  focus  on 
functions  related  to  this  thesis. 

The  cortex  has  subdivisions  particular  to  specific  functions:  somatosensory,  language, 
vision,  auditory  processing,  etc.  Each  of  these  is  further  subdivided  into  specialized  functional 
areas  that  are  connected  to  each  other  in  a  reciprocal  manner  (if  A  goes  to  B  then  B  goes  to 
A)  [16].  Dr.  Mumford  divides  the  connections  into  three  types  of  pathways. 

•  Ascending  pathways:  going  from  a  higher  area  (level  that  is  more  abstracted  from  the 
sensory  data  or  immediate  motor  commands)  to  a  lower  area  (level  that  is  more  directly 
connected  and  related  to  sensory  data  or  immediate  motor  commands). 

•  Descending  pathways:  going  from  lower  to  higher  areas. 

•  Additional  descending:  going  from  higher  areas  to  lower  areas  where  the  difference  in 
abstraction  is  not  readily  apparent  [16]. 
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The  ascending  pathways  allow  detailed  information  about  stimuli  to  be  abstracted  into  more 
general  concepts  and  into  the  ‘language’  of  the  higher  area.  Dr.  Mumford  hypothesizes  the 
descending  pathways  translate  their  hypothesis  of  what  is  happening  back  to  the  lower  area, 
in  its  language,  as  a  template  to  allow  comparison,  so  that  hypotheses  may  be  competed. 
He  also  hypothesizes  that  the  additional  descending  pathways  might  carry  residual,  or  un¬ 
explained  portions  of  the  sensory  data,  to  allow  a  more  global  (outside  the  sending  area’s 
scope)  hypothesis  to  be  generated  [16].  This  would  be  analogous  to  asking  an  equal  what 
their  opinion  is  for  something  you  cannot  explain. 

The  cortico-cortical  loops  can  be  modeled  based  on  Dr.  Mumford’s  architecture,  using 
standard  artificial  neural  networks  for  connections  in  a  looping  fashion.  The  higher  area  can 
create  an  image  of  the  lower  level’s  data  by  making  a  conglomeration  of  templates  (like  placing 
templates  of  trees  on  top  of  a  template  of  a  deer)  and  handing  it  back  to  the  lower  level  for 
comparison.  The  lower  level  will  compare  and  excite  the  higher  level  in  areas  that  are  missing 
in  the  conglomeration,  and  inhibit  areas  that  are  extraneous  (existing  in  the  conglomeration, 
but  not  in  the  sensory  information) .  This  looping  action  will  allow  convergence  on  a  solution 
to  the  question,  “What  am  I  looking  at?” 

The  basic  computing  elements  in  the  cortex  are  the  cortical  columns  [7].  These  columnar 
groups  of  neurons  were  reported  by  Vernon  B.  Mountcastle,  in  the  somatosensory  cortex, 
and  by  David  H.  Hubei  and  Torsten  N.  Wiesel,  in  the  visual  cortex  [5].  Estimates  of  the  size 
vary  in  species,  as  well  as  cortical  section,  in  the  range  50  (im  to  500  fim  [20]. 

2.2  Time  Perception 

Perception  is  psychosomatic.  That  is,  perception  is  all  in  your  head.  Visual,  auditory,  tactile, 
and  taste  perception  are  just  a  few  of  the  illusions  that  humans  make  for  themselves,  to  allow 
them  to  interpret  the  sensory  input  that  they  are  bombarded  with.  Along  the  same  lines,  time 
is  a  confabulation  of  the  brain  that  allows  us  to  bind  sensory  information  from  one  instance 
to  another.  While  this  may  seem  a  presumptuous  statement,  the  work  of  Dr.  Benjamin  Libet 
lends  substantial  credence  [10,  11]. 

Figure  2.2  shows  the  results  of  Dr.  Libet’s  experiments  concerning  subjective  referral  of 
time  [11].  Dr.  Libet  reports  that,  while  a  delay  of  approximately  500  milliseconds  occurs  be¬ 
fore  conscious  perception  of  the  stimuli,  the  conscious  experience  of  the  stimuli  is  subjectively 
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referred  back  in  time  to  approximately  the  actual  stimulus  [10,  11],  The  AER  signal  is  the 
cortical  response  to  a  stimulis,  discussed  in  section  4.3,  page  27. 


0  100  -200  300  400  500  msec 


experience 

reported 


Figure  2.2:  Subjective  Referral  in  Time  of  a  Conscious  Event  [10] 

Dr.  Libet’s  experimental  results  open  a  1/2  second  window  in  which  external  influences 
have  an  opportunity  to  affect  the  perception  of  a  previous  stimulus  [11].  A  profound  example 
of  external  influences  affecting  perception  is  Geldard  and  Sher rick’s  cutaneous  rabbit  illu¬ 
sion  [4].  In  this  illusion,  a  number  of  taps  (approximately  5)  are  delivered  at  regular  intervals 
to  the  wrist  followed  by  additional  taps  further  up  the  arm  without  breaking  rhythm  (40- 
GO  msec  between  pulses)  [4].  The  subject  perceives  a  series  of  evenly  spaced  taps  reminiscent 
of  a  rabbit  hopping  from  the  location  of  the  first  taps  to  the  location  of  the  final  taps  [4]. 
While  it  is  interesting  that  the  first  series  of  taps  seems  to  affect  the  perceived  location  of 
later  series’  taps  (suggesting  that  perception  uses  prior  information  to  shape  current  per¬ 
ception),  the  critical  indication  is  that  subsequent  series’  taps  appear  to  affect  the  perceived 
location  of  previous  taps.  This,  in  turn,  suggests  that  the  later  taps  occurred  prior  to  the 
perception  of  the  previous  taps,  giving  the  appearance  of  non-causal  processing.  In  other 
words,  the  second  series  of  taps  causes  taps  from  the  first  series  to  be  perceived  as  moving 
in  the  direction  of  the  second  series’  location.  If  perception  were  immediate,  or  even  faster 
than  10’s  of  milliseconds,  then  the  brain  would  not  know  which  direction  to  4 move’  the  first 
series’  taps. 
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2.3  Memory  Processes 


Memory  appears  to  have  different  forms  including,  but  not  limited  to:  immediate,  short-term, 
long-term,  and  recognition  memory  [14].  As  there  is  a  vast  quantity  of  information  in  current 
circulation  about  memory  it  is  important  to  describe  only  those  forms  that  are  relative  to 
this  thesis.  The  thesis  is  concerned  with  immediate  and  short-term  memory,  because  they 
seem  to  be  required  for  perception  of  motion. 

At  the  lowest-level,  immediate  memory  of  what  you  are  sensing  as  the  world  changes, 
seems  to  be  a  function  of  habituation. 

Among  biological  mechanisms  that  can  encode  temporal  information,  is  a  par¬ 
ticularly  simple  and  well-understood  phenomenon  known  as  habituation.  Primar¬ 
ily,  habituation  is  a  means  by  which  biological  neural  systems  vary  their  synaptic 
strengths  in  order  to  ignore  repetitive,  irrelevant  stimuli.  Habituation  serves  as  a 
novelty  filter.  [21] 

While  section  2.1,  page  6,  discussed  the  connection  between  the  amygdala,  hippocampus, 
and  the  ability  to  store  new  long-term  memories,  the  storage  location  is  central  to  the  AFIT 
view  of  memory.  The  model  views  memory  as  the  processing  of  information,  and  therefore, 
memory  is  expected  to  be  stored  in  the  processing  areas  of  the  cortex.  Doctors  Mishkin 
and  Appenzeller  report  that  the  most  likely  location  for  memories  is  the  cortical  areas  where 
sensory  impressions  take  shape  [14]. 

The  sub-cortical  memory  circuits  must  therefore  engage  in  a  kind  of  feedback 
with  the  cortex.  After  a  processed  sensory  stimulus  activates  the  amygdala  and 
hippocampus,  the  memory  circuits  must  play  back  on  the  sensory  area.  That 
feedback  presumably  strengthens  and  so  perhaps  stores  the  neural  representation 
of  the  sensory  event  that  has  just  taken  place.  [14] 

2.4  Conclusions 

Mankind  has  acquired  a  great  deal  of  knowledge  regarding  the  brain’s  basic  structure  and 
function.  Correlations  are  made  between  the  actions  of  certain  areas  of  the  brain,  and  their 
observable  effects.  There  is  evidence  of  brain  function  found  in  large  scale,  due  to  damaged 
brain  structure  and  large  area  electrical  recording.  The  damage  and  recordings  are  compared 
with  observable  behavioral  changes,  like  the  ability  to  lay  down  the  new  memories  required 
to  learn  new  tasks,  or  the  ability  to  recall  old  memories  and  use  them  to  recognize  family 
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members.  This  thesis  effort  seeks  to  leverage  on  this  knowledge  and  advance  the  use  of  the 
AFIT  multi-electrode  brain  chip  to  help  discover  some  of  the  remaining  unknowns: 

•  What  is  the  form  data  takes  on  in  the  brain? 

•  Exactly  how  is  data  processed? 

•  How/where  is  data  stored  and  retrieved? 
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3.  Brain  Chip  Development  and  Testing 


AFIT’s  brain  chip  is  a  tool  for  direct  measurement  of  electroencephalogram  (EEG)  data 
at  the  cortical  surface.  It  was  designed  to  allow  observers  to  view,  electrically,  the  cerebral 
cortex  at  approximately  the  cortical  column  scale  [7].  The  brain  chip  presently  allows  for  256 
spatially  separated  measurements  in  an  area  of  less  than  a  dime. 

The  actions  recorded  in  this  section  achieve  thesis  goal  number  1,  page  3. 

3.1  History 

The  brain  chip  effort  began  with  a  4  x  4  array  of  electrodes  that  was  placed  directly  on  the 
visual  cortex  of  a  laboratory  beagle  (Canis  familiaris)  in  1982  [7].  The  successful  retrieval  of 
electroencephalogram  (EEG)  and  visually  evoked  response  (VER)  data  energized  the  devel¬ 
opment  of  improved  brain  chips. 

James  Reid’s  1993  thesis  details  the  early  history  of  the  AFIT  brain  chip  from  Joseph  Tat- 
man’s  1979  16-electrode  design,  that  was  not  successfully  fabricated,  through  Gary  Fitzger¬ 
ald’s  redesign  and  successful  fabrication,  to  George  German’s  effort  to  passivate  the  chip  [19]. 
This  work  allowed  Russell  Hensley  and  David  Denton,  under  the  tutelage  of  Dr.  Kabriski,  to 
implant  a  chip  in  ‘Ricky’,  a  laboratory  beagle  [19].  Reid  goes  on  to  document  improvements 
in  design,  including  his  work  on  the  16x16  array  chip,  referred  to  herein  as  the  reference 
chip  [19]. 

There  are  currently  two  designs  that  have  been  implemented  and  tested:  the  reference  chip 
and  the  differential  chip.  The  chips  were  fabricated  through  the  Metal  Oxide  Semiconductor 
Implementation  Service  (MOSIS)  using  the  2.0jura-wall  process.  Twenty  four  die  of  each 
design  were  returned  with  4  die  packaged  in  40-pin  dual  in-line  packages  (DIPs). 

3.2  Related  Work 

Much  work  has  been  done  on  cortical  systems,  however,  the  AFIT  brain  chip  focuses  on 
recording  two  dimensional  cortical  electoencephalographic  data  at  the  scale  of  the  cortical 
column. 

In  1966,  DeMott  reported  his  work  on  a  toposcopic  technique  for  studying  the  electrical 
activity  of  the  cortex  [1].  DeMott  bundled  400  wires  into  an  array  with  1.5  millimeter  on- 
center  spacing,  and  placed  them  directly  on  the  cortex  of  monkeys  and  cats,  among  others  [1]. 
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DeMott  successfully  recorded  steep  voltage  gradients  (up  to  1  mV /mm)  [1].  To  make  his 
measurements  as  precise  as  possible,  DeMott  used  400  individual  differential  (to  eliminate 
any  common-mode  noise)  amplifiers  giving  him  a  usable  range  of  input  signals  from  50  to 
500/zVpP[l].  DeMott ’s  measurements  were  taken  from  the  visual  cortex,  to  record  visual 
evoked  responses.  The  peak  to  peak  voltages  were  typically  in  the  range  200-500  /jVpp ,  and 
showed  repeated  patterns  in  response  to  identical  stimuli  [1,  20]. 

Dr.  Walter  Freeman  has  collected  EEG  data  from  the  olfactory  cortex  of  rabbits  using 
approximately  90  electrodes,  spaced  500  \xm  apart  [3].  Dr.  Freeman  recorded  signals  which 
could  be  correlated  with  ‘recognized5  scents  that  exhibited  bursts  of  high  frequency  (20-90  Hz) 
activity  with  an  amplitude  of  approximately  100  fiVpp  [3]. 


3.3  Reference  Chip 

The  reference  chip,  seen  in  figure  3.1,  is  so  named  because  it  takes  one-sided  measurements 
relative  to  the  electrical  potential  of  an  L-shaped  electrode. 

3.3.1  Design. 

The  goals  to  be  met  by  the  reference  chip  are  shown  in  Table  3.1  [19].  To  meet  the 
requirements  concerning  a  limited  number  of  input  and  output  lines,  the  array  is  scanned, 
and  output  is  multiplexed  onto  a  single  line. 


Table  3.1:  Requirements  for  Reference  Chip  [19] 


Requirement 

Reason 

Implemented 

Small  size 

Approximately  match  the  size  of 
individual  cortical  columns 

dime  sized 

Bidirectional  current  flow 

Allow  both  stimulation  and  record¬ 
ing  of  the  cortex 

Yes 

Small  number  of  I/O  connections 

Difficulty  in  making  connections 
from  inside  the  cranium  to  external 
circuitry 

7 

Number  of  I/O  connections  not 
dependent  on  the  number  of 
electrodes 

Allows  the  number  of  electrodes  to 
be  increased  without  increasing  the 
number  of  connections 

Multiplexing 

Capability  to  individually 
address  the  electrodes 

Allows  stimulation  and  recording  to 
individual  locations  of  the  cortex 

Yes 

The  reference  chip  (figure  3.2)  consists  of  256  electrodes  arranged  in  a  16  x  16  array,  two 
4  to  16  decoders,  sixteen  transmission  gates,  an  8-bit  counter,  and  an  L-shaped  reference 
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Figure  3.1:  Photograph  of  Reference  Chip 
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pad  [19].  One  4  to  16  decoder  uses  the  least  significant  bits  of  the  8-bit  counter  to  select  one 
of  the  16  transmission  gates  to  drive  the  multiplexed  output  line,  scanning  each  column.  The 
second  4  to  16  decoder  is  connected  to  the  most  significant  bits  of  the  8-bit  counter,  selecting 
a  row  to  be  scanned  [19].  The  electrodes  are  spaced  evenly  at  250  jim  on  center,  which  is  the 
average  of  estimates  for  cortical  column  size  given  in  chapter  2.1,  page  8. 

A  substantial  portion  of  the  design  and  implementation  work  is  discussed  in  the  passiva¬ 
tion  section,  section  3.5,  page  23. 

3.3.2  Testing. 

Each  lot  of  chips  (reference  and  differential)  received  from  the  manufacturer  was  tested 
for  validity  of  design,  accurate  implementation  of  the  design  and  proper  function  prior  to  en¬ 
tering  the  time  consuming  encapsulation  process  (refer  to  section  3.5,  page  23).  Additionally, 
this  testing  includes  discovery  of  properties  of  the  implemented  design  that  are  required  to 
adequately  assess  any  results  from  tests  conducted  after  implantation.  These  properties  are 
the  source  impedance  (amount  of  impedance  between  the  sensor  pad  currently  connected  via 
the  multiplexor  to  the  output  line  and  the  multiplexor  output  line)  and  the  leakage  noise  (the 
effect  that  the  leakage  current  from  a  voltage  potential  on  one  sensor  pad  has  on  neighboring 
pads) . 

Verification  of  design  and  proper  function  was  performed  by  placing  a  marker  voltage, 
per  figure  3.3,  and  taking  measurements  of  the  multiplexed  output  line.  The  marker  voltage 
allowed  the  oscilloscope  to  be  triggered  consistently  in  sync  with  scanning.  Proper  operation 
of  the  counters  was  verified  by  changing  the  time  base  on  the  oscilloscope  to  allow  an  observer 
to  count  the  number  of  clocks  between  marker  signals  (256  clocks/marker).  To  make  certain 
that  every  row  and  column  is  scanned,  the  marker  was  placed  on  each  pad  of  the  main 
diagonal  and  output  was  verified  with  the  oscilloscope.  The  output  row  sync  performance 
was  validated  with  the  oscilloscope  in  much  the  same  manner  as  counting.  The  row  sync 
output  was  seen  every  16  clock  cycles. 

Figure  3.3  shows  where  sources  were  placed  and  data  were  acquired  directly  to  and  from 
the  chip’s  sensor  pads,  using  a  microprobe  station.  ‘Marker’  shows  where  a  5  Vdc  source  was 
placed,  on  the  first  row  and  column  to  allow  the  oscilloscope  to  be  triggered  at  a  known  time. 
Vtest  is  where  an  alternating  current  source  was  placed  to  test  the  ability  of  the  referenced 
brain  chip  to  sample  low  voltage  alternating  current  signals,  like  those  expected  on  the  cortical 
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surface.  And,  Vnoise  is  where  a  probe  was  placed  to  measure  leakage  from  the  neighboring 
pad. 


Figure  3.3:  Microprobe  placement  for  analog  testing  of  the  referenced  brain  chip 

The  source  impedance  causes  a  voltage  difference  between  the  sensor  pad  and  the  output 
line,  because  of  voltage  division  between  the  source  impedance  and  the  the  impedance  between 
the  output  signal  and  ground.  Figure  3.4  shows  the  circuit  for  this  test,  as  well  as  the  method 
for  producing  the  marker  peak  for  all  other  tests.  Using  Ohm’s  Law,  equation  3.1,  a  value  for 
the  combined  impedance  of  the  first  and  second  (row  and  column)  transistors,  responsible 
for  selecting  each  pad,  can  be  calculated  as  approximately  33  kQ.  Further  accuracy  can  be 
achieved  by  careful  measurement  of  the  voltages  and  resistances  that  are  accessible. 


Figure  3.4:  Circuit  of  source  impedance  measurement 


Ohm’s  Law: 

V (voltage)  —  i(current)  *  R(resistence )  (3.1) 

can  be  rearranged  to  read 


i  =  V/R  (3.2) 

Using  the  sampled  and  measured  output  value  of  the  brain  chip,  0.22  U,  and  the  2  kQ 
shunting  resistor, 
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(3.3) 


•  0.22  V"  mi  4 

i  =  — — —  =  0.11  mA 

2kn 


Now,  re-substituting  into  equation  3.2  for  the  entire  circuit: 


0.11  mA  = 


bV 


10  kQ  +  RsD\  +  Rsd2  +  2  kQ 


Therefore: 


(3.4) 


RsD\  +  Rsd2  ~  33  kQ  (3.5) 

This  value  is  reasonably  low,  allowing  the  use  of  a  small  shunting  resistor,  without  voltage 
dividing  the  signal  to  immeasurable  levels. 

Leakage  noise  was  measured  via  the  circuit  shown  in  figure  3.3.  Noise  was  apparent 
and  cycled  every  256  samples  (corresponding  to  the  neighboring  sensor  pad).  However, 
placing  a  2  kQ  resister  between  the  signal  and  ground,  to  simulate  resistance  on  the  cortical 
surface,  removed  all  signs  of  this  noise  signal.  Additionally,  the  source  voltage  for  the  noise 
measurements  was  approximately  1,000  times  larger  than  the  largest  expected  on  the  brain. 
This  source  was  most  likely  capacitively  coupling  with  the  neighboring  pads.  Therefore,  we 
do  not  expect  to  effect  the  brain’s  electrical  activity. 

The  next  test  made  certain  that  a  row  sync  was  output  from  the  chip  every  16th  clock 
cycle.  The  oscilloscope  showed  this  signal  was  pronounced  and  as  expected. 

The  final  test  placed  a  signal  on  the  chip  which  was  systematically  reduced  to  simulate  the 
frequency  (5-40  Hz)  and  magnitude  (10-200  fiVpp )  expected  on  the  cortical  surface  (Vtest  from 
figure  3.3).  This  initial  series  only  tested  down  to  370  iiVpp,  at  150  Hz.  At  this  point,  very 
high  frequency  noise  becomes  apparent.  There  is,  additionally,  a  standing  noise  wave  riding 
on  the  signal  (55  mVpp),  associated  with  clock  ticks  (see  figure  3.7,  page  22,  for  the  nearly 
identical,  but  much  smaller  noise  signal  of  the  differential  chip.).  This  noise  is  substantially 
reduced  from  a  higher  voltage  by  changing  the  rise  time  on  the  input  clock  to  approximately 
5 //seconds.  Additionally,  bandpassing  the  amplifier  at  lk  ~  30kHz  allows  amplification 
with  a  gain  of  500  without  saturating  the  amplifier.  The  standing  wave  is  not  a  problem  for 
sampling  because  sampling  is  done  at  regular  intervals,  however,  the  marker  voltage  must  be 
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reduced  to  the  same  amplitude  as  the  noise  to  avoid  saturation.  At  this  point,  the  oscilloscope 
is  unable  to  trigger  on  the  marker,  and  further  tests  become  impossible. 

Testing  of  the  reference  chip  was  halted  to  begin  tests  on  the  dilferential  chip,  due  to 
the  inability  to  sync  the  oscilloscope.  The  differential  chip  allowed  for  common  mode  noise 
rejection,  which  has  the  effect  of  reducing  the  standing  wave,  allowing  higher  amplification. 

Additional  noise  reduction  methods  used  to  get  this  far  are:  placing  2  kQ  shunting  resisters 
from  both  output  and  row  sync  to  ground,  to  draw  current;  reduction  of  Vu  to  3.1  V. 

3.4  Differential  Chip 

The  differential  chip,  figure  3.5,  gains  its  name  from  the  dual  output  channels  to  be  used  for 
differential  amplification. 


Figure  3.5:  Photograph  of  Differential  Chip 


3.^.1  Design. 

The  differential  chip  was  designed  with  the  same  requirements  as  the  referenced  chip,  as 
well  as  requirements  to  improve  the  signal  to  noise  ratio  (Table  3.2).  The  differential  chip 
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accomplishes  this  in  two  ways:  differential  measurement  and  dividing  the  array  into  two 
sub-arrays. 


Table  3.2:  Requirements  for  Differential  Chip  [19] 


Requirement 

Reason 

Implemented 

Small  size 

Approximately  match  the  size  of 
individual  cortical  columns 

dime  sized 

Bidirectional  current  flow 

Allow  both  stimulation  and  record¬ 
ing  of  the  cortex 

Yes 

Small  number  of  I/O  connections 

Difficulty  in  making  connections 
from  inside  the  cranium  to  external 
circuitry 

7 

Number  of  I/O  connections  not 
dependent  on  the  number  of 
electrodes 

Allows  the  number  of  electrodes  to 
be  increased  without  increasing  the 
number  of  connections 

Multiplexing 

Capability  to  individually 
address  the  electrodes 

Allows  stimulation  and  recording  to 
individual  locations  of  the  cortex 

Yes 

Reduced  Noise 

Allow  amplification  of  the  signal 
to  make  precision  at  least  20  /jV 

Yes 

The  differential  chip  consists  of  two  8x17  sub-arrays,  as  shown  in  figure  3.6.  In  this 
design  two  neighboring  electrodes,  row-wise  paired,  are  connected  to  two  separate  output 
lines,  simultaneously.  These  two  lines  are  then  compared  using  a  differential  amplifier  to 
remove  common  mode,  or  identical,  noise.  This  greatly  improves  the  signal  to  noise  ratio 
because  any  noise  that  one  electrode  (or  it’s  path  to  the  amplifier)  picks  up,  the  identical 
noise  should  be  picked  up  by  the  second  electrode.  Subtracting  one  from  the  other  removes 
this  noise  and  leaves  any  unique  portion  of  the  signal  unaffected. 

Dividing  the  array  into  two  sub-arrays  improves  signal  to  noise  ratio  by  allowing  the  clock 
to  run  at  1/2  the  frequency  required  for  the  reference  chip.  Because  each  sub-array  has  its 
own  output  lines,  and  both  sub-arrays  are  controlled  by  the  same  clock,  the  differential  chip 
outputs  two  samples  to  every  one  sample  from  the  reference  chip.  Running  the  clock  slower, 
allows  a  greater  time  span  to  sample  the  multiplexed  output  for  each  chip  sample.  The  data 
collection  section  (section  4.4,  page  29)  describes  how  averaging  multiple  samples  improves 
signal  to  noise  ratio. 

The  differential  chip  provides  a  reset  instead  of  a  row  sync,  as  provided  by  the  reference 
chip.  The  reset  allows  the  tester  to  synchronize  the  scanning  of  the  array.  The  row  sync 
did  not  allow  synchronization  of  the  whole  array,  it  only  allowed  synchronization  to  the  row 
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scans.  The  row  sync  left  the  tester  to  wonder  which  pad  was  being  connected  to  the  output 
line  at  any  one  time.  This  is  not  the  case  with  the  reset. 

3,4*  %  Testing . 


Figure  3.7:  Oscilloscope  Traces  of  Output  Signal  from  Differential  Chip  (Bandpass  Filter: 
300  Hz  -  10  kHz) 


The  differential  chip  was  tested  in  the  exact  same  manner  as  the  reference  chip,  to  the 
point  that  the  reference  chip  testing  was  halted  due  to  noise.  The  differential  chip  showed 
a  substantial  reduction  in  the  standing  noise  signal,  caused  by  the  clock.  This  allowed 
amplification  with  a  gain  of  2000,  compared  to  500  for  the  reference  chip.  An  additional 
factor  in  the  ability  to  test  with  lower  input  values  on  the  differential  chip,  is  the  availability 
of  the  reset.  The  reset  allowed  the  oscilloscope  to  be  triggered  without  a  marker  voltage. 

Two  resisters  (10  k£l  and  50  Q)  where  placed  in  series  with  the  input  signal  to  perform  a 
voltage  division  of  200.  The  divided  signal  was  used  as  Vtest •  Vtest  was  lowered  to  20  fiVPPi  and 
15  jjVpp  was  measured  at  the  output.  This  implies  a  gain  on  the  brain  chip  of  0.75,  however, 
this  is  not  consistent  with  readings  at  higher  voltages  that  imply  a  gain  of  0.28.  The  gain  of 
0.28  is  what  is  considered  correct,  because  at  20  fiVpp  the  signal  to  noise  ratio  is  approximately 
1  (Vnoise  «  20  fiVpp),  and  precise  readings  from  an  oscilloscope  become  improbable.  The 
amplifier  settings  used  were:  gain  =  2000  and  bandpass  was  300  77 £  —  30  kHz. 

The  high  frequency  nature  of  the  noise  found  at  20  fiVPP)  figure  3.8  allows  oversampling 
and  averaging  to  improve  the  signal  to  noise  ratio,  and  increase  the  precision.  The  signal  is 
oversampled  by  a  factor  of  16.  Because  averaging  increases  the  signal  by  N2  and  the  noise 
by  VN,  the  precision  gain  is  expected  to  be  64.  The  estimated  precision,  where  signal  to 
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Figure  3.8:  Oscilloscope  Traces  of  Output  Signal  from  Differential  Chip  Containing  Approx¬ 
imately  20  ptVpp  of  High  Frequency  Noise 


noise  is  expected  to  be  1  after  averaging,  is  0.3  f %VPP .  This  will  allow  measurement  of  voltage 
differences  over  the  gambit  of  levels  recorded  by  other  researchers. 

During  the  experiment,  all  voltages  measured  must  be  scaled  by  the  chip  gain,  to  get 
accurate  estimations  of  the  voltage.  However,  it  is  the  relative  activity,  and  not  the  exact 
voltages  that  is  of  interest. 

3.5  Passivation 

Passivation  of  the  brain  chips  is  required  to  protect  the  chips  in  the  semiconductor-hostile 
environment  of  the  brain  [19].  The  brain  is  bathed  in  cerebral  spinal  fluid  (CSF),  consisting 
primarily  of  sodium  ions,  7Va+,  and  potassium  ions,  K+ .  There  are  two  steps  in  passivation: 
electrode  protection  and  chip  encapsulation. 

3.5.1  Electrode  Protection. 

Protecting  the  metal  pads  from  the  CSF  requires  the  use  of  a  conducting  material  that 
does  not  react  in  this  environment.  Because  MOSIS  only  provides  aluminum  metalization,  the 
AFIT  brain  chips  must  be  plated  with  such  a  non-reactive  metal.  Work  has  been  performed 
at  AFIT  with  gold  and  iridium  [19].  For  this  thesis,  chips  where  coated  with  either  gold  or 
iridium.  The  full  process  for  preparing  the  electrodes  to  be  coated  with  metal  can  been  found 
in  Appendix  C. 

The  processes  of  both  gold  and  iridium  metalization  are  identical,  except  for  power  set¬ 
tings  and  timing  information  specified  in  Appendix  C.  Both  metals  require  a  layer  of  titanium 
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on  top  of  the  aluminum  pads  to  ensure  a  good  electrical  connection  through  the  aluminum 
oxide  layer  that  always  builds  up  on  exposed  aluminum.  Gold  is  commonly  used  for  sensing 
devices,  however,  iridium  carries  higher  current  densities.  This  makes  iridium  preferable 
for  stimulation  applications.  An  additional  consideration  is  that  the  iridium  was  easier  to 
process. 

Additionally,  an  ultrasound  cleaner  was  used  to  clear  the  excess  gold.  The  ultrasound 
cleaner  worked  extremely  well,  and  without  apparent  damage  to  the  surface,  as  with  the 
cotton  swab  method.  However,  during  later  processing,  all  of  the  sensor  pads  fell  off.  The 
ultrasound  works  because  the  metal  vibrates  differently  than  the  silicon,  loosening  the  excess 
metal,  as  well  as  the  aluminum  pads. 

The  iridium  coating  lifted  off  without  excessive  use  of  force,  and  appears  to  be  the  better 
coating. 

3.5.2  Encapsulation. 

The  metallized  chips  must  be  protected  from  the  brain  fluid  on  all  areas  not  metallized. 
DuPont  Pyraline  polyimide  is  a  relatively  easy  substance  to  use,  and  meets  the  requirement 
of  passivation  [19].  The  full  process  for  polyimide  coating  the  chips  is  in  Appendix  C. 

The  polyimide  coating  process,  while  not  complex,  requires  some  practice.  It  is  more 
artistic  than  exacting.  Etching  the  polyimide  can  be  painstakingly  slow,  however,  over¬ 
etching  can  occur  suddenly.  Any  over-etched  chips  can  be  cleaned  and  reused. 

3.6  Final  Packaging 

The  passivated  chips  must  be  packaged  for  implantation  and  interfaced  with  the  laboratory 
measurement  equipment.  To  meet  these  ends,  the  chip  is  glued  to  a  header,  “■  •  -a  microwave 
device  package  manufactured  by  the  Airpax  corporation”  [19].  Wires  are  bonded  from  the 
input  and  output  pads  on  the  chip,  and  the  entire  package  is  coated  with  polyimide,  excepting 
the  exposed  electrode  array.  All  wire  connections  are  made  to  the  pins  extending  from  the 
back  of  the  header.  The  wire  connections  were  protected  and  strengthened  with  rubber 
cement. 
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3.7  Conclusions  and  Recommendations 


The  AFIT  differential  chip  is  a  superior  brain  chip  to  the  reference  chip.  The  differential  chip 
has  proven  itself  as  an  effective  measuring  device  capable  of  measuring  differential  voltages 
of  20 1 iV  for  visual  reading  from  an  oscilloscope,  and  has  a  reset  that  is  extremely  useful  for 
test  purposes. 

The  high  frequency  nature  of  the  noise  found  at  20  allows  oversampling  and  averaging 
to  improve  the  signal  to  noise  ratio,  and  increase  the  precision.  The  estimated  precision,  where 
signal  to  noise  is  expected  to  be  1  after  averaging,  is  0.3  fiV. 

Two  differential  chips  were  successfully  metallized  with  gold  and  passivated.  These  chips 
were  wire  bound  and  fully  prepared  for  implantation. 
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4 .  Methodology 


4.1  Introduction 

Achieving  thesis  goals  2-4,  page  3,  required  substantial  preparation,  test  execution,  and  data 
acquisition  and  analysis.  This  section  explains  the  approach  taken. 

4.2  Implantation  and  Testing 

The  actual  implantation  effort  was  planned  and  reported  in  the  attached  protocol,  Ap¬ 
pendix  B. 

4-2.1  Test  Instrumentation. 


Figure  4.1:  Test  Equipment  Set  Up 

Five  pieces  of  equipment  constitute  the  test  suite:  AFIT  differential  brain  chip,  Harvard 
Research  differential  amplifier,  National  Instruments  AT-MIO-16E-2  data  acquisition  (DAQ) 
board  (PC  insert),  2  relays,  40  Vdc  power  supply,  and  2  cylinoids.  The  National  Instruments 
board  was  installed  in  a  486  PC-clone  equipped  with  16  Mbytes  of  random  access  memory 


and  was  augmented  with  shielded  cabling  and  a  National  Instruments  SCB-68  68-pin  shielded 
connector  block  to  reduce  noise. 

The  DAQ  board  sends  10  mseconds  width  pulses,  that  activate  the  relays  (either  #1  or 
#2),  closing  a  circuit  with  the  40  Vdc  power  supply  and  cylinoids  #1  or  #2,  which  provides  a 
sharp  tap  for  tactile  stimulation.  The  DAQ  board  simultaneously  initiates  data  acquisition. 
Voltage  differences  on  the  cortex  are  measured  between  neighboring  sensor  pads  on  the 
AFIT  brain  chip.  The  output  of  neighboring  pads  are  referenced  to  ground  and  amplified 
differentially  by  the  Harvard  Research  differential  amplifier  (band  width:  300-30  kHz;  gain: 
2000).  The  amplified  signal  is  fed  single-ended  to  the  DAQ  board,  where  it  is  sampled. 

Power,  ground,  and  clock  for  the  brain  chip  are  provided  by  the  DAQ  board.  All  signals 
are  directly  tied  to  the  DAQ  board  ground  except  for  the  output  of  the  Harvard  Research 
differential  amplifier.  For  this  reason,  the  DAQ  board  is  programmed  to  measure  the  dif¬ 
ference  between  the  amplifier  output  and  the  amplifier’s  reference,  without  reference  to  the 
DAQ’s  ground.  This  measurement  method  provides  common  mode  rejection  for  any  noise 
picked  up  between  the  amplifier  and  the  DAQ  board. 

To  insure  that  the  input  to  the  differential  amplifier  will  not  be  overloaded  by  a  static 
charge  on  the  test  subject,  the  subject  must  be  grounded.  This  is  accomplished  with  an 
electrode  placed  behind  the  ear  of  the  test  subject.  Grounding  from  locations  farther  from 
the  brain  increases  the  chance  of  introducing  noise  from  the  cardio-pulmonary  system. 

4*3  Development  of  Test  Scenarios 

There  are  two  test  scenarios  developed  and  reported  herein:  averaged  evoked  response 
(AER),  and  apparent  motion.  The  AER  test  scenario  was  developed  to  show  that  the  AFIT 
array  could  take  usable  electro-encephalogram  measures  from  a  living  animal’s  cerebral  cor¬ 
tex.  The  apparent  motion  test  scenario  was  developed  to  use  the  AFIT  array’s  capability  to 
record  cortical  activity  during  what  is  subjectively  an  apparent  motion  illusion  for  human’s, 
on  the  cortex  of  a  living  animal. 

4-3.1  Averaged  Evoked  Response  (AER)  Test. 

The  sensory  evoked  response  is  the  voltage  signal  that  can  be  recorded  on  the  cortex 
after  and  corresponding  to  a  stimulus.  Dr.  Libet  refers  to  the  averaged  evoked  response, 
the  algebraic  average  of  multiple  identical  stimulus-recording  data  sets,  because  a  single  SER 
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recording  has  a  very  low  signal  to  noise  ratio  [11,  20].  For  all  experiments  herein,  the  AER  is 
obtained  by  averaging  recorded  responses  to  50  identical  stimulations.  The  expected  results 
can  be  seen  in  figure  4.2,  with  a  somewhat  lower  SNR  because  we  are  using  50  stimulations, 
versus  256  per  Libet  [11].  Additional  variation  can  be  expected  because  this  thesis  reports 
measurements  taken  from  the  cortex  of  a  rhesus  monkey,  not  a  human,  as  in  figure  4.2 

0  100  ‘200  300  400  500  msec 
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Figure  4.2:  Averaged  Evoked  Response  in  Human,  Using  256  Instances  [11] 

Because  stimulation  is  performed  in  an  analogous  manner  to  Dr.  Libet’s,  the  large  peak 
response  is  expected  at  approximately  100  ms  after  the  stimulus. 

4-3.2  Apparent  Motion  Test  Setup . 

The  apparent  motion  test  is  based  on  Geldard  and  Sherrick’s  cutaneous  rabbit  illusion, 
reported  in  section  2.2,  page  8.  To  recreate  the  illusion,  where  rhythmic  tactile  stimulations 
at  discrete  locations  feels  like  evenly  spaced  stimulations  (hopping  rabbit),  two  identical 
cutaneous  stimulators  are  used  [4].  These  are  the  same  stimulators,  as  used  in  the  AER 
scenario. 

There  are  three  sequences  of  stimuli  required  for  this  scenario,  one  illusory  and  two  control 
sequences.  The  illusory  sequence  consists  of  two  stimuli  of  2  msec  duration  and  50  msec 
separation  at  location  1,  followed  by  3  stimuli  of  equal  duration  and  separation,  to  maintain 
rhythm,  at  location  2.  The  two  control  sequences  consist  of  identical  stimuli  and  timing, 
however,  one  maintains  all  taps  at  location  1  and  the  other  maintains  them  at  location  2.  The 
controls  will  allow  post  processing  to  remove  any  signal  information  related  to  position  of 
stimuli  and  timing  of  repeated  stimuli,  leaving  only  the  apparent  motion  artifacts  (assuming 
that  there  are  any  to  be  recorded) . 
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4.4  Data  Collection 


Data  is  collected  during  all  test  scenarios  in  the  same  manner.  The  N ational  Instruments  DAQ 
board  drives  the  brain  chip  with  a  20  kHz  clock  signal,  and  acquires  16  evenly  spaced  samples 
per  clock  cycle,  for  a  sample  rate  of  320  kHz.  16  samples  are  taken  per  clock  cycle  so  that  they 
can  be  averaged  to  increase  the  measurement  system’s  signal  to  noise  ratio,  removing  high 
frequency  noise  introduced  by  multiplexing  and  amplifying  the  measures  brain  activity  (see 
figure  4.3).  Because  averaging  increases  the  signal  by  N2 3  and  the  noise  by  y/N,  the  precision 
gain  is  expected  to  be  64.  The  estimated  precision,  where  signal  to  noise  is  expected  to  be  1 
after  averaging,  is  0.3  \iVvv.  This  should  not  be  confused  with  the  averaging  done  to  improve 
the  signal  to  noise  ratio  of  the  sensory  evoked  response  (noise  from  within  the  brain) . 


Figure  4.3:  Over-sampling  the  Multiplexed  Output 

Data  is  recorded  in  this  fashion  for  a  half  a  second  for  each  experiment  iteration.  At  the 
end  of  this  time  period  the  1/2  second’s  worth  of  data  is: 

1.  Averaged  in  increments  of  16,  to  reduce  SNR  as  stated  earlier,  and  then  demultiplexed. 
This  reduces  the  amount  of  data  samples  from  159,744  to  9984  (approximately  1/2 
second  at  20  ksamples/sec) 

2.  Demultiplexed  into  128  different  sensor-related  signals,  representing  1/2  of  the  chip. 
This  places  the  9984  samples  into  128  series  each  containing  78  data  points  for  the  1/2 
second.  This  meets  the  Nyquist  criteria  for  sampling  frequencies  up  to  78  Hz. 

3.  Stored  to  disk. 

Appendix  A  contains  the  C-code  responsible  for  initializing  the  hardware,  controlling  the 
experiments,  demultiplexing  and  compiling  the  results.  Additionally,  the  program  allows  a 
sensor  to  be  chosen  for  display  of  a  running  average  response. 
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Experience  running  this  program  on  a  486  has  shown  that  the  minimum  memory  required 
is  16  Mbytes.  At  8  Mbytes  the  hard  disk  is  required  for  virtual  memory  and  the  performance 
plummets. 

4,5  Conclusions 

Completed  preparation  for  experimentation  required  successful  implementation  and  testing 
of  the  software  used  to  control  the  National  Instruments  data  acquisition  boards;  successful 
design  and  development  of  the  measurement  and  amplification  circuitry;  and  successful  design 
and  development  of  the  stimulation  system. 

The  central  theme  of  this  design  is  the  National  Instruments  board,  which  allowed  all 
activity  to  be  synchronized.  All  portions  of  this  section  were  successfully  implemented  and 
tested.  Additionally,  the  ability  to  oversample  increases  the  effective  precision  to  0.3  jiV. 
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5.  Results  and  Conclusions 


5.1  Surgery 

Dr.  James  R.  Cooper,  AL/OEVM,  performed  the  surgery  on  an  adult  rhesus  monkey  on 
25  October,  1995.  The  monkey  had  a  terminal  cancer  and  was  to  be  euthanized  that  same 
day.  The  monkey  was  under  general  anesthesia  for  the  entire  experiment  (halethane  and 
nitrous  oxide),  and  was  euthanized  immediately  following  the  experiment.  The  procedure 
successfully  exposed  the  gyrus  postcentralis  (see  figure  5.1),  where  the  somatosensory  cortex 
is  expected  to  exist. 
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Figure  5.1:  Labeled  Baboon  Brain,  Used  to  Locate  the  Gyrus  Postcentralis  [22] 

During  the  experiment,  two  chips  were  used  and  data  were  recorded  for  all  experiments 
with  each  chip.  In  both  cases,  bubbles  were  observed  emanating  from  beneath  the  chip 
shortly  after  contacting  the  brain.  Additionally,  a  substance  that  appeared  to  be  polyimide 
was  observed  coming  from  under  the  chips.  After  completion  of  each  experiment,  damage 
was  observed  in  the  brain  tissue  immediately  under  the  chip.  The  damage  appeared  to  be 
a  burn.  The  monkey’s  brain  was  sent  for  an  autopsy,  and  the  subsequent  results  will  shed 
additional  light  on  the  EEG  recordings  taken. 

The  most  probable  cause  of  the  coating  problems  is  that  a  different  polyimide,  PI-2722, 
was  unknowingly  used,  with  the  same  process  as  for  the  polyimide  used  in  past  work  at 
AFIT.  The  cure  temperature  was  not  high  enough  to  meet  recommendations  for  PI-2722. 
All  instruments  and  references  have  been  updated  in  this  thesis  to  avoid  this  mistake  in  the 
future. 
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5.2  Results 


Data  were  collected  corresponding  to  tests  with  both  chips.  All  tests  were  conducted:  aver¬ 
aged  evoked  response  (1  test  series  with  each  stimulator),  apparent  motion,  apparent  motion 
controls  (2  test  series  using  same  sequence  as  apparent  motion,  but  all  taps  being  delivered 
at  the  same  location).  For  the  first  chip,  after  the  first  15  iterations  (out  of  250),  a  substantial 
signal  was  introduced.  This  signal,  figure  5.2,  is  evidently  caused  by  the  clock.  The  second 
chip  experienced  the  same  problem  after  25  iterations. 


Figure  5.2:  Clock  Noise  Measured  with  Second  Brain  Chip — 78  samples  over  500  msec 

After  approximately  30  minutes,  the  clock  noise  on  chip  1,  reduced  and  disappeared.  The 
chip  was  functional  through  the  entire  period,  and  after  the  noise  died  down,  the  signal  in 
figure  5.10  was  recorded. 

All  plots  contained  within  this  report  show  a  time  sequence  of  78  samples  taken  in  1/2 
second.  The  vertical  scale  is  in  volts.  Because  of  clock  noise,  none  of  the  apparent  motion 
data  is  considered  valid. 

5. 2  A  Averaged  Evoked  Response. 

Figure  5.3  shows  plots  of  the  EEG  signal  at  one  of  the  sensor  locations  on  the  second 
chip.  The  top  plot  shows  a  single  instance  (iteration)  of  EEG  data,  while,  the  bottom  shows 
the  average  of  15  iterations  at  the  same  sensor  location  as  in  the  top  plot. 

Figure  5.4  shows  the  averaged  evoked  response  observed  in  data  from  the  second  chip. 
The  plot  was  obtained  by  averaging  first  across  iterations  1  thru  25  of  the  AER  experiments, 
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Figure  5.3:  Single  Sensor  Measurement:  Sensor  40,  with  single  iteration  15  on  top  and  average 
of  iterations  1  to  15  on  bottom — 78  samples  over  500  msec 


Figure  5.4:  Recorded  Averaged  Evoked  Response:  First  Chip,  Average  of  All  Sensors  Over 
15  Iterations — 78  samples  over  500  msec 
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and  then  averaging  across  all  128  sensor  readings.  As  is  evident,  the  signal  is  less  pronounced 
than  that  in  figure  5.5,  coming  from  only  15  samples. 


Figure  5.5:  Recorded  Averaged  Evoked  Response:  Average  of  All  Sensors  Over  25  Iterations 
with  Second  Chip — 78  samples  over  500  msec 

Figure  5.5  shows  the  averaged  evoked  response  observed  in  data  from  the  second  chip. 
The  plot  was  obtained  by  averaging  first  across  iterations  1  thru  25  of  the  AER  experiments, 
and  then  averaging  across  all  128  sensor  readings.  The  signal  to  noise  ratio  is  poor,  due 
predominantly  to  the  small  data  set  used.  The  plot  shows  a  wave  with  an  amplitude  swing 
of  approximately  150  LiVpp,  and  a  period  for  the  half  cycle  shown  of  1/2  second.  To  compare 
this  signal  with  figure  4.2,  page  28,  the  main  peak  occurs  approximately  190  msec  after  the 
stimulus  instead  of  100  msec.  However,  they  are  similar  in  that  a  dip  immediately  precedes 
the  peak  in  both  cases. 

5.2.2  Phase  Reversal 

To  further  characterize  the  performance  of  the  array,  the  average  of  all  sensors  in  the  top 
1/2  of  the  chip  is  compared  to  the  bottom  1/2,  using  the  same  data  set  used  for  figure  5.5. 
The  resulting  plot,  figure  5.6,  shows  substantial  phase  reversal.  The  most  obvious  conclusion 
to  be  drawn  is  that  the  source  of  the  signal  lies  roughly  along  the  center  line  between  the  top 
and  bottom  halves  of  the  chip. 
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Figure  5.6:  Recorded  Averaged  Evoked  Response:  Average  of  All  Sensors  Over  25  Iterations 
with  Second  Chip  (Top  vs.  Bottom) — 78  samples  over  500  msec 


Figure  5.7:  Recorded  Averaged  Evoked  Response:  Average  of  All  Sensors  Over  25  Iterations 
with  Second  Chip  (Right  vs.  Left) — 78  samples  over  500  msec 
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Figure  5.7  shows  another  view  of  the  same  data.  However,  in  this  case,  the  array  is 
divided  into  left  and  right  halves.  Once  again,  there  is  substantial  phase  reversal,  indicating 
a  signal  source  roughly  centered  on  the  chip. 


Figure  5.8:  Recorded  Averaged  Evoked  Response:  Average  of  All  Sensors  Over  25  Iterations 
with  Second  Chip  (Every  Other) — 78  samples  over  500  msec 

Figure  5.8  shows  yet  another  view  of  the  same  data  set.  This  view  takes  all  of  the  even 
numbered  sensor  locations  and  plots  them  against  the  odd  numbered  sensors,  as  seen  in 
figure  5.9.  The  substantial  phase  reversal  correlates  with  the  concept  of  measuring  at  the 
cortical  column  level.  That  is,  if  signals  are  being  produced  at  centers  (cortical  columns) 
spaced  approximately  the  same  distance  as  the  electrodes  we  would  expect  phase  reversal  at 
the  individual  electrode  level. 

5.2,3  Post  Noise  Saturation  Period. 

Figure  5.10  shows  data  recorded  with  the  first  chip,  after  the  period  of  noise  saturation. 
What  appears  to  be  an  evoked  response  is  clearly  visible  («  100  fiVpp)}  with  a  substantially 
improved  signal  to  noise  ratio.  This  signal  cannot  be  an  artifact  of  other  activity,  because  it 
is  synchronized  with  the  presentation  of  a  stimulus.  The  stimulus  was  not  presented  at  an 
exact  interval  because  of  the  Microsoft  Windows  interrupt  driven  environment. 
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Figure  5.9:  Every  Other  Sensor  Selection 


Figure  5.10:  Recorded  Signals  After  Noise  Saturation—78  samples  over  500  msec 


5.3  Conclusion 


The  differential  brain  chip  has  proven  its  ability  to  record  data  from  the  cortex.  While  there 
were  problems  that  kept  the  chip  from  recording  apparent  motion  signals,  the  chip  continued 
to  work  and  allowed  recording  of  data  after  the  coating  failed.  It  is  apparent  that  the  spacing 
of  the  sensor  pads  is  appropriate,  at  least  for  rhesus  monkeys,  to  measure  at  the  cortical 
column  scale. 

The  entire  test  setup  worked,  excepting  the  failure  of  the  chip  passivation,  and  allowed 
collection  of  substantial  data.  A  pronounced  AEG  has  been  reported,  and  additional  in¬ 
formation  lies  within  the  recorded  data,  as  yet,  undiscovered.  Additional  work  that  can  be 
accomplished  with  the  data  involves  toposcopic  views  of  the  activity  in  the  cortical  area  that 
was  directly  under  the  chip  and  the  relation  of  the  extremely  low  noise  signal  discovered  after 
damage  was  done  to  the  cortex  to  the  actual  autopsy  findings. 

Work  must  be  done  to  improve  the  passivation  so  that  the  chip  does  not  harm  the  brain, 
as  was  the  case  in  this  experiment. 
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Appendix  A:  C-Code  for  Test  Scenario  Control 

/ /  ****  Brain  Chip  Data  Acquisition  Program  ****  // 

#include  <windows.h> 

#include  <conio.h> 

#include  <stdio.h> 

#include  <math.h> 

#include  Mwdaq_bc.hM 
#include  "nidaqcns .hM 

#define  num_channels  16  //  Sample  each  mux  cycle  num_channels  times 

#define  gain  1  //  Gain  of  input  amplifier 

#define  num_pads  128  //  Number  of  sensors  in  mux  signal 

#define  num_mux_samps  9984  //  Approx  .5  sec.  at  20kHz 

#define  num„dmux_ samps  78  //  What’s  left  after  downsampling  and  demux 

#define  num.samples  159744  //  num_mux_s amps *num_ channels 

#def ine  samp_int  62  //  f loor( l/(mux_f req*#channels*int_clk_res) ) 

#define  out _r ate  100  //  Rate  of  output 

int  experiment;  //  Counter  for  which  of  5  experiments  to  run 
int  iteration;  //  How  many  times  an  experiment  has  run 
int  Error;  //  Records  error  codes 
int  i,j,k;  //  Generic  counters 

long  1;  //  Long  counter  to  use  with  huge  arrays 
int  sync;  //  Used  to  align  the  demuxed  signals 
double  sync_value;  //  Used  to  align  the  demuxed  signals 
double  max;  //  Used  to  display  plot 

HANDLE  hsampbuf f er=0 ;  //  Handle  for  the  sample  buffer 
int  huge  *ipsampbuff er ;  //  Big  pointer  to  sample  buffer  (for  locked) 
HANDLE  hvoltbuff er=0;  //  Handle  for  the  pre-demuxed  voltage  levels 
double  huge  *ipvoltbuff er ;  //  Big  pointer  (for  locked) 

HANDLE  hsetlout=0;  //  Handle  for  set  1  out  buffer 

int  huge  *ipsetlout;  //  Big  pointer  to  above  (for  locked) 

HANDLE  hset2outl=0;  //  Handle  for  set  2  outl  buffer 

int  huge  *ipset2outl;  //  Big  pointer  to  above  (for  locked) 

HANDLE  hset2out2=0 ;  //  Handle  for  set  2  out2  buffer 
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int  huge  *ipset2out2;  //  Big  pointer  to  above  (lor  locked) 

float  huge  data[128]  [78] ;  //  Holder  for  data  to  put  to  file 

float  cum_data[78] ;  //  Cumulative  data  to  plot 

double  accumd;  //  Used  to  average  voltages 

float  accum;  //  Used  for  other  accumulations 

int  stopped;  //  Whether  or  not  the  DAQ  stopped 

unsigned  long  samples;  //  #  samples  actually  collected 

int  out_ch0=0;  //  Output  on  DACO 

int  out_chl=l;  //  Output  on  DAC1 

double  set lout [3] ={-.5, 5, -.5};  //  Experiments  1  &  2 
double  set2outi [10] =  //  Experiments  3,  4  &  5,  1st  sequence 
{-.5, 5, -.5, -.5, -.5, -.5, 5, -.5, -.5, -.5}; 

double  set2out2[13]=  //  Experiments  3,  4  &  5,  2nd  sequence 
{-.5, 5, -.5, -.5, -.5, -.5,5, -.5, -.5, -.5, -.5, 5, -.5}; 

int  in_chan_vct [num_ channels] =  //  Vector  defining  the  sequence  to  scan 
{0,0, 0,0, 0,0, 0,0, 0,0, 0,0, 0,0, 0,0}; 

int  gain_vector [num_channels]=  //  Vector  of  associated  gain  settings 
{gain , gain , gain , gain , gain , gain , gain , gain , 
gain , gain , gain , gain , gain , gain , gain , gain} ; 

FILE  *FID;  //  File  to  store  data  in 

//  ***  initialize  Settings  on  the  NI-DAQ  Board  ***  // 
void  initialize(void) 

{ 

//  ***  Initialize  Input  Portion  of  the  Board  ***  // 

if (Error=AI_Conf igure(l ,0,2, 0,0,0) )  //  AI_0  is  single  ended, 

printf (MAI_Conf igure  failed  7*d", Error);  //  Nonref erenced  and  bipolar 

if (Error=DAQ__Config(l ,  1,2))  //  Set  for  external  trigger 

printf  (MDAC]_Conf  ig  failed  ,/,dH  ,  Error ) ;  //  and  ext.  scan-interval 

if  (Error=Select_Signal ( 1 , ND_IN_SCAN_START , //  Scan-interval  set  to 

ND_PFI_9,NDJL0W_T0_HIGH))  //  utilize  Freq_0ut  for 

printf ("Scan-interval  not  set  %d", Error);  //  synchrony 

if  (Error=Select_Signal(l ,ND_IN_START_TRIGGER,//  Use  stimulus  on 

ND_PFI_0 ,  ND__L0W_T0_HIGH ) )  //  ferret  as  trigger 

printf  ("Trigger  not  set  #/*d"  , Error ) ; 
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if (Error=Select_Signal ( 1 , ND_FREQ_OUT , / /  Freq_Out  set  to  20kHz  to 
ND_INTERNAL_100__KHZ , 5) )  //  drive  brainchip  and  scan- 
printf ("Freq_0ut  not  set  Xd" , Error) ;  //  interval 

//  ***  Clear  Output  Channels  to  Avoid  Spurious  Levels  ***  // 
if (Error=A0_VWr ite ( 1 , 0 , - . 5 ) )  //Clear  output  channel  0 
printf ("Clearing  output  failed  %d" , Error) ; 
if (Error=AO_VWrite(l, 1,-. 5))  //Clear  output  channel  1 
printf ("Clearing  output  failed  %d" , Error) ; 

> 

int  main (void) 

{ 

USE_E_Series ;  //  just  bring  in  usable  functions 

//  ***  Set  Up  Buffers  for  Data  Transfers  to  and  from  NI-DAQ  Board  ***  // 
hsampbuf f er= (HANDLE) GlobalAlloc (GMEM_M0VEABLE , / /  Allocate  an  acq. 

(DWORD ) s izeof (int)*num_ samples ) ;  //  buffer  for  DMA 
ipsampbuf f er=(int  huge  *)GlobalLock (hsampbuf f er) ;  //  Lock  it 
hvoltbuffer= (HANDLE) GlobalAlloc (GMEM.MOVEABLE,//  Allocate  a  voltage 
(DWORD )sizeof (double )*num_samples) ;  //  buffer  for  conversion 
ipvoltbuff er= (double  huge  *)GlobalLock(hvoltbuff er) ;  //  Lock  it 
hs et lout= (HANDLE) Global Alloc (GMEM_M0VEABLE , / /  Allocate  an  Output 
(DWORD )sizeof (int) *3) ;  //  buffer 

ipsetlout=(int  huge  *)GlobalLock(hsetlout ) ;  //  Lock  it 
hset2outl= (HANDLE) GlobalAlloc (GMEM_M0VEABLE , / /  Allocate  an  Output 
(DWORD)sizeof (int)*6) ;  //  buffer 

ipset2outl=(int  huge  *)GlobalLock(hset2outl) ;  //  Lock  it 
hset2out2=(HANDLE)GlobalAlloc(GMEM_M0VEABLE,//  Allocate  an  Output 
(DWORD)sizeof (int)*9);  //  buffer 

ipset2out2=(int  huge  *)GlobalLock(hset2out2) ;  //  Lock  it 

//  ***  Set  Up  Basic  Configuration  ***  // 
initializeQ  ; 
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//  ***  Load  Test  Output  Buffers  ***  // 
if (Error=WFM_Scale(l,0,3,l,//  Convert  voltages  to  binary 
setlout ,ipsetlout))  //  values  for  output  sequence  set  1 
printf  ("Output  Scaling  Failed  */,d"  , Error)  ; 
if (Error=WFM_Scale (1,0,10,1,//  Convert  voltages  to  binary 
set2outl , ipset2outl) )  //  values  for  output  sequence  set  1 
printf  ("Output  Scaling  Failed  '/,d" , Error) ; 
if (Error=WFM„Scale (1,0,13,1,//  Convert  voltages  to  binary 
set2out2, ipset2out2) )  //  values  for  output  sequence  set  1 
printf  ("Output  Scaling  Failed  */,d" , Error)  ; 

//  ***  Go  Through  All  Five  Experiments  ***  // 
f  or  (experiments ;  experiments ;  experiment++) 

{ 

switch (experiment+1) 

{ 

case  1:  if (Error=Select_Signal(l,//  Use  stimulus  (pos#0)  on 

ND_IN_START„TRIGGER, / /  ferret  as  trigger 

ND_PFI_0 , ND_L0W_T0_HIGH) ) 

printf  ("Trigger  not  set  '/,d"  , Error)  ; 

FID=f open("expl .txt" , "w") ;  //  Create  file  to  save  to 
break ; 

case  2:  if (Error=Select_Signal(i,//  Use  stimulus  (pos#l)  on 

ND_IN_START_TRIGGER , / /  ferret  as  trigger 

ND_PFI_1 , ND_L0W  JT0_HIGH) ) 

printf  ("Trigger  not  set  '/,d"  , Error) ; 

fclose(FID);  //  Close  previous  file 

FID=fopen("exp2.txt" , "w") ;  //  Create  file  to  save  to 

break; 

case  3:  if (Error=Select_Signal(l ,//  Use  stimulus  (pos#0)  on 

ND_DLSTART_TRIGGER, / /  ferret  as  trigger 

ND_PFI_0 , ND_LQW_TQ„HIGH ) ) 

printf  ("Trigger  not  set  */,d"  , Error) ; 

fclose(FID);  //  Close  previous  file 

FID=fopen("exp3.txt","w") ;  //  Create  file  to  save  to 
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break; 

case  4:  if (Error=Select_Signal(l,//  Use  stimulus  (pos#0)  on 

ND_IN_START_TRIGGER , / /  ferret  as  trigger 

ND_PFI_0 , ND_LOW_TO_HIGH ) ) 

printf  ("Trigger  not  set  '/,d" , Error) ; 

fclose(FID);  //  Close  previous  file 

FID=f open("exp4 . txtM , "w" ) ;  //  Create  file  to  save  to 

break; 

case  5:  if (Error=Select_Signal(l,//  Use  stimulus  (pos#l)  on 

ND_IN_START_TRIGGER , / /  ferret  as  trigger 

ND_PFI_1 , ND_L0W_T0_HIGH) ) 

printf  ("Trigger  not  set  #/,d"  , Error ) ; 

f close(FID) ;  //  Close  previous  file 

FID=fopen("exp5.txt","w") ;  //  Create  file  to  save  to 

break; 

> 

//  ***  Do  Each  Experiment  50  times  ***  // 
f or(iteration=0 ; iteration<50 ; iteration++) 

{ 

//  ***  set  the  Trigger  on  the  Analog  Input  ***  // 
if (Error=SCAN_Setup(l ,num_channels , //  Init  scan  vectors: 
in_chan_vct ,gain_vector) )  //  scan  AI_0  repeated 
printf  ("Scan  setup  failed  '/,d" , Error) ;  //  set  scan  sequence 
if (Error=SCAN_Start ( 1 , ipsampbuf f er , / /  Set  scan  sequence: 
num_samples,-3,samp_int,0,0))  //  ext.  scan  sync 
printf  ("Scan  setup  failed  */,d"  ,  Error )  ;  //  int .  samp  timebase 

//  ***  Stimulate  the  Test  Subject  ***  // 
switch(experiment+l) 

{ 

case  1:  if (Error=WFM_0p(l , l,&out_ch0,//  Send  Seq  1  on  DACO 
ipsetlout,3,l,out_rate))  //  For  Experiment  1 
printf ("Output  Scaling  Failed  Xd" , Error) ; 
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break ; 

case  2:  if  (Error=WFM_Op(l ,  l,&out__chl ,//  Send  Seq  1  on  DAC1 
ipsetlout ,3, 1 ,ont_rate) )  //  For  Experiment  2 
printf  ("Output  Scaling  Failed  #/,d" , Error) ; 
break; 

case  3:  if (Error=WFM_Op(l , 1 ,&out_chO ,//  Send  Seq  2  on  DACO 

ipset2outl, 10, 1 ,out_rate) )  //  For  Experiment  3 

printf  ("Output  Scaling  Failed  #/#d" , Error) ; 

if (Error=WFM_Op(l, 1 ,&out_chl ,//  Send  Seq  2  on  DAC1 

ipset2out2,13,l,out_rate))  //  For  Experiment  3 

printf  ("Output  Scaling  Failed  */#d" , Error) ; 

break; 

case  4:  if (Error=WFM„Op(l , 1 ,&out_chO,//  Send  Seq  2  on  DACO 

ipset2outl , 10 , 1 ,out_rate) )  //  For  Experiment  4 

printf  ("Output  Scaling  Failed  y#d",Error); 

if (Error=WFM_Op(l, 1 ,&out_chO,//  Send  Seq  2  on  DACO 

ipset2out2,13,l,out_rate))  //  For  Experiment  4 

printf ("Output  Scaling  Failed  Xd" , Error) ; 

break; 

case  5:  if (Error=WFM_Op(l , 1 ,&out_chl ,//  Send  Seq  2  on  DAC1 

ipset2outl,10,l,out_rate))  //  For  Experiment  5 

printf  ("Output  Scaling  Failed  */,d" , Error) ; 

if (Error=WFM_Op(l, 1 ,&out_chl ,//  Send  Seq  2  on  DAC1 

ipset2out2, 13 , 1 , out_rate) )  //  For  Experiment  5 

printf  ("Output  Scaling  Failed  */d",  Error); 

break; 

> 

//  ***  Process  the  Results  ***  // 

do  Error=DAQ_Check(l,&stopped,&samples) ;  //  Is  it  soup  yet? 
while ( ! stopped) ; 

if (Error=DAQ_VScale ( 1,0, gain ,1,0,//  Convert  to  voltage 
num_ samples , ipsampbuff er , ipvoltbuff er) ) 
printf  ("Conversion  Failed  #/,d", Error); 
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//  ***  Demultiplex  the  Results  ***  // 

1=0; 

for(i=0; i<nunudmux_samps ; i++)  //  Separate  each  time  sample 
for(j=0; j<num_pads; j++)  //  Separate  each  sensor  pad 

{ 


accumd=0; 

f or (k=0;k<num_ channels ;k++)  //  Average  the  oversampled 
{  //  signal  to  increase  SNR 
accumd+=ipvoltbuffer [1] ; 

i++; 


> 

> 


data[j]  [i] = (float )accumd/num_channels ; 


//  ***  Align  the  demuxed  signals  so  that  pads  are  in  order  ***  // 
sync=0;  //  Initialize  the  sync 
sync_value=10000;  //  Initialize  the  sync  value 
f or (i=0 ; i<num_pads ; i++)  //  Check  each  pad's  signal 
{ 

accum=0 ; 

f or( j=0; j<10 ; j++)  //  Get  change  in  first  10 
accum+=f abs(data[i] [j]-data[i]  [j  +  1]  ) ;  //  time  samples 
if (accum<=sync_value) 

{ 

sync=i; 

sync„value=accum ; 

> 

} 


//  ***  Write  the  Results  to  disk  ***  // 
f or (i=0 ; i<num_dmux_ samps ; i++)  //  Go  through  time 
{ 

accum=0 ; 

for(j=sync; j<num_pads; j++)  //  Go  through  each  sensor  pad 
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{ 

fprintf (FID, "%f  " ,data[j]  [i] ) ;  //  Write  to  disk 
accum+=data[j]  [i] ;  //  Total  them  up  to  display 
> 

for(j=0; j<sync; j++)  //  Do  the  rest 

{ 

fprintf  (FID ,  "'/.f  "  ,data[j]  [i]  ) ;  //  Write  to  disk 
accum+=data[j] [i] ;  //  Total  them  up  to  display 
> 


fprintf (FID, "\n") ;  //  Put  each  on  it’s  own  line 

cum_data[i] +=(accum/num_dmux„ samps) ; //  Average  all  sensors 


} 


//  ***  Control  the  display  ***  // 
if (experiment ! =0)  //  If  not  first  experiment 
printf  ("experiment  */*d  iteration  */,d  \n",//  Tell  where  we  are 
(experiment+l) , (iteration+1) ) ; 
else  //  Otherwise 
{ 

//  ***  Display  cumulative  results  ***  // 
max=0 ; 

f or(i=0; i<num_dmux_ samps ; i++)  //  Find  greatest  and  least 

{ 

if (max<fabs(cum_data[i]  ) )max=f abs(cum_data[i]  ) ; 

> 

clrscr();  //  Clear  the  screen 
f or(i=0; i<num„dmux_ samps ; i++)  //  Plot 
{ 

gotoxy(i, (12-(int) (ll*(cum_data[i] /max)) ) ) ; 
printf ("*"); 

} 

> 


}  //  End  of  Iteration  FOR  Loop 
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}  //  End  of  Experiment  FOR  Loop 


//  ***  Clear  Output  Channels  to  Avoid  Spurious  Levels  ***  // 
if (Error=A0_VWrite(l,0,-.5))  //Clear  output  channel  0 
printf  ("Clearing  output  failed  */.d"  , Error) ; 
if (Error=A0_VWrite(l,l,-.5))  //Clear  output  channel  1 
printf  ("Clearing  output  failed  */,dM ,  Error) ; 

//  ***  Free  Up  Memory  ***  // 

GlobalUnlock(hvoltbuffer) ;  //  Unlock  volt  buffer 
GlobalFree(hvoltbuff er) ;  //  Give  it  back 
GlobalUnlock(hsampbuff er) ;  //  Unlock  sample  buffer 
GlobalFree(hsampbuff er) ;  //  Give  it  back 
GlobalUnlock(hsetlout) ;  //  Unlock  out  buffer 
GlobalFree(hsetlout) ;  //  Give  it  back 
GlobalUnlock(hset2outl) ;  //  Unlock  out  buffer 
GlobalFree(hset2outl) ;  //  Give  it  back 
GlobalUnlock(hset2out2) ;  //  Unlock  out  buffer 
GlobalFree(hset2out2) ;  //  Give  it  back 
fclose(FID);  //  Close  last  file 
return  0; 

> 
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Appendix  B:  Protocol 


ANIMAL  USE  PROTOCOL  COVER  SHEET 

1.  Title:  A  Multielectrode,  Multiplexed  Silicon  Cortical  Electrode 

2.  Purpose  of  the  Study: 

To  test  in  vivo  the  performance  of  a  multielectrode,  multiplexed  cortical  electrode.  This 
16  X  16  array  (256  total)  electrode  system  is  designed  to  both  measure  naturally  occurring 
distributed  cortical  activity  and  to  insert  two-  dimensional  data  into  the  cortex.  In  the  later 
mode  it  could  serve  as  a  visual  prosthesis  for  blindness  caused  by  lesions  distal  to  the  cortical 
mapping  of  visual  data.  The  electric  field  measurements  taken  from  the  surface  of  the  brain 
are  called  electroencephalogram  (EEG)  tracings  which  represent  the  mean  excitatory  state 
of  the  group  of  neurons  lying  directly  beneath  the  measuring  electrode.  The  performance  of 
the  multielectrode  array  will  be  assessed  by  the  ability  of  the  device  to  record  EEG  data. 

3.  DoD  Relevancy: 

Animal  visual  systems  have  certain  capabilities  which  have  direct  military  application. 
Primates  especially  have  visual  systems  with  high  foveal  acuity,  color  discrimination,  and 
gestalt-based  form  perception.  If  such  capabilities  could  be  duplicated  by  machine,  they 
would  have  immediate  direct  application  in  the  automatic  target  acquisition  and  terminal 
guidance  phases  of  smart  missile  systems.  Current  “smart”  weapons  still  require  a  human 
operator  to  find  the  target  and  set  up  a  laser  based  terminal  guidance  system.  Alternatively, 
the  U.S.  Army  has  developed  a  series  of  TV-guided,  anti-tank  missiles  in  which  a  TV  picture 
from  the  missile  is  transmitted  to  a  human  guidance  operator  who  actually  “flies”  the  missile 
by  remote  control. 

No  reliable  automatic  target  locator  or  flight  guidance  system  based  on  visual  form  has 
ever  been  put  into  operational  service  even  though  research  on  this  so  called  “pattern  re¬ 
cognition”  problem  has  been  vigorously  pursued  for  about  forty  years.  It  is  clear  that  there 
are  some  fundamental  capabilities  immanent  in  mammalian  visual  systems  that  we  simply 
do  not  understand;  if  these  naturally  evolved  techniques  could  be  discovered  from  properly 
instrumented  animal  visual  systems,  it  might  be  possible  given  the  current  advanced  state  of 
electronic  fabrication  capabilities,  to  make  a  fundamental  breakthrough  in  weapon  guidance 
systems. 

4.  Type  and  #  of  Animals: 

Species:  Macaca  mulatta  Species:  Mustela  putorius 

Sex:  Male  Sex:  Male 

Weight:  12  KG  Weight:  1  KG 

Age:  12  to  15  years  Age:  12  months 

Total  number  required:  1  Total  number  required:  3 

5.  Disposition  of  Animals: 

The  study  is  designed  to  allow  for  the  animal  to  return  to  the  colony  following  surgical 
removal  of  the  chip.  In  the  event  the  health  of  the  animal  deteriorates  or  for  some  other  reason, 
euthanasia  is  indicated,  the  animal  will  be  administered  sodium  pentobarbital  intravenously 
to  effect. 

6.  Principal  Investigator/Organization/Extension: 

Steven  K.  Rogers,  AFIT/ENG,  255-  6565,  x4284 
Matthew  Kabrisky,  AFIT/ENG,  255-9267. 
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CORTICAL  ELECTRODE  ANIMAL  PROTOCOL 


1.  Title:  A  Multielectrode,  Multiplexed  Silicon  Cortical  Electrode 

2.  Project /Task/ Work  Unit: 

3.  Principal  Investigator: 

Steven  K.  Rogers,  AFIT/ENG 
Matthew  Kabrisky,  AFIT/ENG 

4.  Associate  or  Co-Investigators:  James  R.  Cooper,  AL/OEVM 

5.  Scientific  Objective: 

To  test  in  vivo  the  performance  of  a  multielectrode,  multiplexed  cortical  electrode.  This 
16  X  16  array  (256  total)  electrode  system  is  designed  to  both  measure  naturally  occurring 
distributed  cortical  activity  and  to  insert  two-  dimensional  data  into  the  cortex.  In  the  later 
mode  it  could  serve  as  a  visual  prosthesis  for  blindness  caused  by  lesions  distal  to  the  cortical 
mapping  of  visual  data.  The  electric  field  measurements  taken  from  the  surface  of  the  brain 
are  called  electroencephalogram  (EEG)  tracings  which  represent  the  mean  excitatory  state 
of  the  group  of  neurons  lying  directly  beneath  the  measuring  electrode.  The  performance  of 
the  multielectrode  array  will  be  assessed  by  the  ability  of  the  device  to  record  EEG  data. 

6.  Military  Relevance: 

Animal  visual  systems  have  certain  capabilities  which  have  direct  military  application. 
Primates  especially  have  visual  systems  with  high  foveal  acuity,  color  discrimination,  and 
gestalt-based  form  perception.  If  such  capabilities  could  be  duplicated  by  machine,  they 
would  have  immediate  direct  application  in  the  automatic  target  acquisition  and  terminal 
guidance  phases  of  smart  missile  systems.  Current  “smart”  weapons  still  require  a  human 
operator  to  find  the  target  and  set  up  a  laser  based  terminal  guidance  system.  Alternatively, 
the  US  Army  has  developed  a  series  of  TV-guided  anti-tank  missiles  in  which  a  TV  picture 
from  the  missile  is  transmitted  to  a  human  guidance  operator  who  actually  “flies”  the  missile 
by  remote  control. 

No  reliable  automatic  target  locator  or  flight  guidance  system  based  on  visual  form  has 
ever  been  put  into  operational  service  even  though  research  on  this  so  called  “pattern  re¬ 
cognition”  problem  has  been  vigorously  pursued  for  about  forty  years.  It  is  clear  that  there 
are  some  fundamental  capabilities  immanent  in  mammalian  visual  systems  that  we  simply 
do  not  understand;  if  these  naturally  evolved  techniques  could  be  discovered  from  properly 
instrumented  animal  visual  systems,  it  might  be  possible  given  the  current  advanced  state  of 
electronic  fabrication  capabilities,  to  make  a  fundamental  breakthrough  in  weapon  guidance 
systems. 

7.  Technical  Background: 

The  AFIT  multielectrode  array  design  was  based  on  several  key  concepts: 

•  Information  is  transmitted  in  a  neuron  by  the  generation  of  electrical  impulses  known 
as  action  potentials. 

•  The  electric  field  produced  by  the  activity  of  neurons  can  be  received  by  an  electrode 
resting  on  the  surface  of  the  cortical  tissue. 

•  The  neurons  in  the  cortex  are  grouped  into  cortical  columns,  which  consist  of  several 
hundred  to  several  thousand  neurons  functioning  together  to  perform  a  given  task. 
These  cortical  columns  are  believed  to  be  the  smallest  functional  elements  in  the  cortex. 
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•  There  is  virtually  no  transmission  of  data  across  the  cortical  sheet.  The  cortical  columns 
are  accessed  primarily  through  input/output  axons  running  through  the  cerebral  alba 
and  between  the  cortex  and  the  brain  stem. 

•  Analysis  of  the  interconnecting  matrix  between  cortical  columns  may  provide  new  in¬ 
sight  to  the  function  of  the  brain. 

Research  similar  to  that  being  proposed  by  AFIT  relies  on  the  use  of  macroelectrodes 
arranged  as  arrays  of  either  intracortical  probes  or  surface  probes.  Intracortical  probes  are 
designed  to  penetrate  the  cortical  surface  in  order  to  record  the  response  of  (or  stimulate) 
small  groups  of  cortical  neurons  completely  in  the  interior  of  the  cortical  column.  Surface 
electrodes,  on  the  other  hand,  are  used  to  record  or  stimulate  the  group  of  neurons  directly  be¬ 
neath  the  electrode.  The  electric  field  measurements  taken  from  the  brain  by  macroelectrodes 
are  called  electroencephalogram  (EEG)  tracings.  An  EEG  represents  the  mean  excitatory 
state  of  the  group  of  neurons  lying  in  close  proximity  to  the  measuring  electrode.  Therefore, 
it  is  reasonable  to  expect  a  surface  electrode  to  record  the  response  of  the  neurons  directly 
below.  Whereas  the  intracortical  probes  allow  studies  primarily  in  the  interior  of  the  cortical 
sheet. 

Properties  of  the  intracortical  probe  concept  emphasize  the  importance  of  getting  more 
intimate  connection  to  the  interior  of  the  cortical  column  system.  They  seem  to  overlook 
the  inevitable  damage  which  these  electrodes  cause.  But  the  fact  that  the  columns  behave 
as  unitary  devices  leads  us  to  assume  that  individual  columns  can  be  effectively  stimulated 
from  the  cortical  surface  ends  of  the  columns  as  in  fact  has  been  done  for  decades.  Therefore, 
the  AFIT  electrode  is  placed  on  the  cortical  surface  where  it  will  produce  little  or  no  trauma 
to  the  cortex  beneath  it  and  where  it  will  still  have  access  to  the  columns  for  recording  and 
stimulation  purposes. 

Early  research  using  arrays  of  surface  electrodes  relied  primarily  on  the  use  of  bundles 
of  very  fine  wires  placed  on  the  surface  of  the  cortex.  In  1966  DeMott  reported  on  the  use 
of  a  400  electrode  array  made  up  of  400  closely  spaced  (  0.25  mm  center-to-center)  wires. 
DeMott ’s  results  obtained  from  a  variety  of  animals,  demonstrated  substantial  differences 
in  the  behavior  of  virtually  adjacent  cortical  areas.  This  behavior  is  consistent  with  the 
existence  of  distinct  functional  cortical  columns  described  by  Mountcastle  (1957)  and  Hubei 
and  Wiesel  (1962). 

In  1968  Brindley  published  the  description  of  a  multielectrode  visual  prosthesis  which  he 
had  implanted  in  the  primary  visual  cortex  of  a  blind  human  volunteer.  This  electrode  system 
used  80  separate  wires  arranged  in  an  approximate  8  X  10  array;  each  wire  was  brought  out 
of  the  subjects  skull  and  connected  to  a  multi-pin  electric  connector.  Electrical  stimulation 
of  any  one  of  the  electrodes  produced  the  subjective  experience  of  “seeing”  a  point  of  light 
in  a  specific  location  of  the  subject’s  visual  field.  Brindley  termed  these  “phosphenes”  and 
considered  them  as  stimulatable  pixels.  Thus  by  stimulating  them  in  appropriate  combinations 
he  could  transmit  crude  geometrical  images  to  the  visual  system  of  a  blind  person.  Needless- 
to-say,  electrical  stimulation  of  the  brain  causes  no  physical  discomfort  to  the  subject  since 
there  are  no  pain  receptors  in  the  cerebral  cortex. 

In  1974,  Dobelle  reported  replication  of  this  procedure  and  produced  a  demonstration 
movie  showing  a  blind  subject  reading  Braille  letters  that  had  been  inserted  into  his  visual 
cortex  by  stimulating  appropriate  sets  of  electrodes.  The  subject  in  Dobelle’s  experiment 


50 


had  been  blind  for  10  years  and  was  able  to  read  Braille  at  30  letters  a  minute  using  a  64 
electrode  array  for  phosphene  generation. 

Semiconductor  technology  overcomes  many  of  the  problems  associated  with  using  wire 
bundle  electrodes.  Wire  electrodes  suffered  from  poor  control  over  their  physical  and  material 
characteristics,  resulting  in  a  lot  of  variation  in  measurements.  Also,  the  number  of  electrode 
sites  was  limited  by  the  amount  of  wires  which  could  be  passed  through  the  scalp.  With 
the  current  semiconductor  technology,  multiplexing  of  the  data  is  easily  performed,  greatly 
reducing  the  number  of  connections  through  the  scalp.  A  few  of  the  devices  made  possible 
by  semiconductor  fabrication  techniques  are  discussed  in  the  following  paragraphs. 

Researchers  at  the  University  of  Michigan  have  designed  a  multichannel  multiplexed  in- 
tracortical  probe.  This  device, which  resembles  a  key,  is  designed  to  be  inserted  into  the 
cortical  tissue.  The  probe  is  typically  15  fi m  in  thickness  with  a  shank  width  as  narrow  as 
20  ^m.  Multiple  electrode  sites  are  exposed  along  the  length  of  the  shank.  The  electrode 
sites  are  multiplexed  together  to  allow  individual  electrodes  to  be  used  for  recording  and  or 
stimulation.  By  attaching  multiple  probes  in  a  precisely  controlled  pattern  to  an  orthogonal 
platform,  a  three-  dimensional  array  of  electrodes  is  possible.  Such  a  three-dimensional  device 
can  be  used  to  monitor  neural  activity  throughout  a  volume  of  cortical  tissue.  However,  use  of 
a  intracortical  probe  does  traumatize  the  tissue.  Bleeding  was  observed  during  experiments 
using  gerbils.  Additionally,  probe  electrodes  were  found  to  have  been  coated  with  biological 
material  which  may  affect  the  recording  capabilities  of  the  electrodes. 

Jones,  et  al.  at  the  University  of  Utah  have  also  developed  a  three-dimensional  intracor¬ 
tical  electrode  array.  This  device  consists  of  a  micro  machined  array  of  100  silicon  “needles” 
on  a  4.2  X  .2  X  0.12  mm  thick  substrate.  The  tip  of  each  sharpened  needle  is  coated  with 
platinum  and  functions  as  the  electrode  site.  The  device  is  being  designed  for  sensory  restor¬ 
ation  via  intracortical  electrical  stimulation  of  a  sensory  cortex.  Experimental  implantation 
of  the  device  has  been  performed  on  cats  with  evidence  of  some  intracortical  bleeding  due  to 
blood  vessel  damage  being  observed.  Also,  only  a  few  electrodes  were  able  to  be  accessed 
with  their  current  technique;  a  demultiplexer  chip  to  allow  access  to  all  electrodes  was  still 
in  development. 

Dagnelie,  et  al.  recorded  visually  evoked  potentials  (VEPs)  in  an  alert  rhesus  monkey 
using  an  array  of  35  electrodes.  The  electrodes  were  organized  into  five  bundles  of  seven 
wires  each,  and  were  implanted  on  the  striate  and  peristriate  visual  cortex.  The  research 
was  performed  as  a  detailed  study  of  cortical  mapping  but  also  demonstrated  the  usefulness 
of  surface  measurements  in  the  study  of  cortical  processing. 

Bartlett  and  Doty  investigated  the  ability  of  monkeys  to  detect  microstimulation  of  the 
striate  cortex.  In  their  experiments,  an  array  of  twelve  electrodes  were  implanted  within 
the  representation  of  central  vision  in  the  visual  cortex  of  a  monkey.  The  animal  was  then 
trained  to  respond  to  the  application  of  0.2  ms  electrical  pulses  at  50  Hz  to  its  striate  cortex. 
The  purpose  of  this  study  was  to  measure  the  threshold  for  the  detection  of  stimulus.  The 
threshold  for  detection  of  the  stimulus  ranged  from  50  to  250  /uA,  depending  on  the  depth 
of  penetration  into  the  cortex.  Bartlett  and  Doty  did  not  test  the  animal’s  response  to  any 
visual  stimulus. 

A  lot  is  known  about  how  the  brain  initially  analyzes  sensory  messages.  Yet,  how  the  brain 
combines  sensory  messages  with  past  experience  for  recognition  is  still  not  known.  Freeman 
believes  that  a  macroscopic  view  must  be  adopted  in  order  to  understand  perception.  His 
research  suggests  that  perception  depends  not  on  the  action  of  individual  neurons,  but  on 
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the  simultaneous,  cooperative  activity  of  millions  of  neurons  spread  throughout  the  cortex. 
Freeman’s  studies  were  concentrated  on  the  neurons  of  the  olfactory  system.  Action  potentials 
from  the  receptor  neurons  in  the  nasal  passages  propagate  to  an  area  of  the  cortex  known  as 
the  olfactory  bulb.  From  there,  new  signals  are  sent  to  many  other  parts  of  the  brain.  Note 
that  this  is  very  similar  to  the  way  visual  signals  are  received  by  the  primary  visual  cortex. 
EEG  data  was  collected  simultaneously  from  60  to  64  electrodes  0.5  mm  apart  attached  to 
the  surface  of  the  olfactory  bulb  of  trained  rabbits. 

In  Freeman’s  experiments,  a  “burst”  of  oscillations  could  be  seen  in  each  EEG  tracing 
when  an  animal  inhaled  a  familiar  scent.  All  the  tracings  from  the  electrode  array  were 
suddenly  more  regular  for  a  few  cycles,  until  the  animal  exhaled.  Also  the  tracings  often 
had  a  higher  amplitude(^100  //V)  and  frequency  (from  20  to  90  Hz).  These  bursts  are 
labeled  as  gamma  waves  denoting  the  frequency  band  of  the  observed  oscillations.  Evidence 
of  a  collective  behavior  was  shown  by  the  presence  of  a  common  waveform,  or  carrier  wave, 
present  in  each  tracing  in  the  sets  of  burst  recordings.  Freeman  found  that  it  is  not  the  shape 
of  the  carrier  wave  that  identifies  an  odorant.  In  fact  the  wave  changed  every  time  the  animal 
inhaled.  However,  when  the  average  amplitude  of  each  electrode’s  carrier  wave  was  plotted 
on  a  grid  representing  the  olfactory  bulb  surface,  a  specific  amplitude  pattern  emerged.  As 
long  as  the  animals  training  was  not  changed  the  same  map  resulted  for  a  particular  odorant, 
even  though  the  carrier  wave  was  different  for  each  sniff. 

Freeman  found  other  parts  of  the  brain  may  exhibit  the  same  chaotic  behavior  seen  in 
the  olfactory  system.  In  fact,  he  has  documented  gamma  bursts  across  large  cortical  regions 
involved  in  the  recognition  of  visual  images.  His  research  suggests  that  familiar  visual  stimuli 
are  also  associated  with  specific  amplitude  maps  of  common  carrier  waves.  Freeman  predicts 
that  when  viewing  a  drawing  in  which  foreground  and  background  is  ambiguous,  such  that 
perception  alternates  between  the  two  images,  the  amplitude  maps  will  be  found  to  alternate 
as  well.  Similar  oscillatory  responses  were  discovered  in  the  cortex  of  cats  and  the  macaque 
monkey  by  Gray  and  Engel. 

It  seems  likely  that  interconnection  systems  between  coupled  arrays  of  cortex  will  be 
multichannel  arrays  of  axial  trunks  interconnecting  two  dimensional  arrays  of  cortical  ele¬ 
ments.  Therefore,  the  two  dimensional  AFIT  array  of  electrodes  should  be  the  best  possible 
arrangement  of  detection  and  stimulation  to  couple  to  these  systems. 

Twelve  years  have  past  since  the  implantation  of  the  first  AFIT  multielectrode  array.  This 
device,  which  consisted  of  sixteen  multiplexed  electrodes  arranged  in  a  four-by-four  array, 
was  implanted  on  the  visual  cortex  of  a  laboratory  beagle  in  1982.  The  device  functioned 
for  fifteen  days  before  being  removed  and  the  subject  ’’Ricky”  recovered  with  no  appar¬ 
ent  side  effects.  This  experiment  validated  the  concept  of  using  multielectrode  arrays  for 
neural  research;  however,  it  also  identified  several  improvements  that  were  necessary.  The 
improvements  included  the  need  for  more  electrodes,  smaller  electrodes  to  better  match  cor¬ 
tical  columns,  better  fabrication  processes  to  improve  the  electrical  characteristics,  and  an 
improved  surgical  and  implantation  procedure.  In  the  years  since  then,  several  thesis  efforts 
have  been  made  to  improve  the  device  design  and  to  solve  the  problems  encountered  with  the 
first  implementation. 

The  current  design  of  the  AFIT  multielectrode  array  is  described  in  detail  by  Rob  Reid 
in  his  1993  thesis  .  This  device  consists  of  256  electrodes  laid  out  in  a  16  X  16  array.  A 
photograph  of  this  device  is  shown  in  Figure  1.  Each  electrode  can  be  enabled  or  disabled 
individually.  An  on-chip  counter  circuit  selects  each  electrode  in  sequence  as  controlled  by 
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an  external  clock  input.  The  electrodes  are  160  X  160  /mi  with  a  center-to-center  spacing  of 
250  /mi.  This  device  was  tested  in  a  saline  environment  simulating  the  environment  of  the 
brain,  and  found  to  function  correctly. 


Figure  1.  The  AFIT  multielectrode  array. 


From  the  physiological  studies,  the  well  known  anatomical  data,  and  the  research  on 
human  volunteers,  it  is  clear  that  it  is  possible  to  insert  subjectively  perceivable  image-like 
stimuli  into  mammalian  visual  systems.  Therefore,  it  is  reasonable  to  assume  that  if  an 
animal  were  trained  to  respond  to  a  specific  visual  stimuli  by,  say,  pressing  a  lever  to  get  a 
food  reward,  then  the  insertion  of  such  a  visual  form  directly  into  its  primary  visual  cortex  by 
electrical  stimulus  should  result  in  the  trained  response.  Such  an  experimental  result  would 
verify  the  integrity  of  the  electronic  and  physiological  systems  as  well  as  the  experimental 
hypothesis  that  direct  stimulation  of  animal  visual  cortex  could  duplicate  the  results  obtained 
in  human  volunteer  subjects. 

Success  at  this  stage  suggests  an  even  more  interesting  extension  of  the  procedure.  While 
damage  to  the  human  primary  visual  cortex  produces  only  blind  spots  in  the  visual  field, 
damage  in  secondary  visual  processing  areas  of  the  cortex  produces  categorical  visual  deficits 
lumped  under  the  general  term:  “agnosias”.  One  of  these,  as  an  example,  prosopagnosia  is 
the  complete  inability  to  recognize  human  faces,  even  ones  own,  while  retaining  otherwise 
normal  visual  perception.  When  a  brain  so  damaged  is  recovered  after  death,  the  only  obvious 
damage  is  the  destruction  of  a  small  piece  of  cerebral  cortex  in  a  secondary  visual  processing 
area.  This  leads  to  postulating  a  system  where  visual  data  are  first  mapped  to  primary  visual 
cortex  where  no  significant  form  analysis  (or  “perception”)  occurs  since  damage  there  only 
causes  circumscribed  blind  spots  in  the  visual  field.  Following  this  mapping,  the  visual  data 
are  transferred  by  cortico-cortical  tracts  in  the  cerebral  alba  to  secondary  visual  areas  in 
the  cortex  where  the  elegant  analysis  we  call  perception  actually  occurs.  Damage  to  these 
secondary  regions  causes  catastrophic  perceptual  disease.  It  would  be  of  great  interest  to 
monitor  primary  and  secondary  visual  areas  simultaneously  in  an  attempt  to  gather  data 
while  the  brain  is  actually  performing  perception. 

Furthermore,  it  is  reasonable  to  expect  that  a  monkey  trained  to  respond  to  a  certain 
stimulus  such  as  a  light  bar  at  a  particular  orientation  could  be  used  to  demonstrate  that  the 
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representation  of  a  stimulus  is  manifested  by  a  particular  gamma  burst.  If  the  response  to 
the  stimulus  was  recorded  and  “played  back”  through  a  stimulating  electrode,  the  monkey 
should  respond  as  trained  just  as  if  the  actual  visual  stimulus  was  given.  Success  of  such  an 
experiment  would  depend  on  the  ability  to  elicit  a  cortex-wide  burst  and  also  serendipitous 
placement  of  the  electrodes. 

8.  Experimental  Design: 

This  experiment  will  consist  of  two  phases.  Phase  one  will  involve  the  use  of  ferrets 
to  verify  that  the  AFIT  array  is  capable  of  recording  EEG  data  from  the  cortex.  Other 
investigators  have  found  that  the  ferret  is  well  suited  for  studies  of  the  visual  system  (Jackson 
and  Hickey,  1985).  Phase  two  will  involve  the  use  of  a  Rhesus  monkey.  The  array  will  be  used 
in  both  species  for  recording  and  stimulation.  Success  of  these  two  phases  will  be  determined 
by  the  ability  of  the  device  to  record  EEG  and  VER  data. 

a.  The  anesthetic  regimen  and  surgical  approach  will  be  the  same  in  both  species.  Prior  to 
surgery,  the  animal  will  be  pre- anesthetized  with  Ketamine  HCL  (15  mg/kg)  and  Atrophine 
(0.04  mg/kg).  Once  anesthetized,  the  head  will  be  shaved  and  prepared  for  sterile  surgery. 
Surgical  anesthesia  will  be  induced  using  a  3-5%  concentration  of  Isofluorane  in  combination 
with  a  80/20  mixture  of  oxygen  and  nitrous  oxide.  The  animals  heart  rate,  respiratory  rate, 
and  blood  pressure  will  be  monitored  during  the  surgical  procedure. 

The  surgical  procedure  will  consist  of  incising  the  skin  and  subcutaneous  tissues  over 
the  visual  cortex.  The  cranial  muscles  will  then  be  bluntly  dissected  exposing  the  skull.  A 
trephine  will  be  used  to  create  a  circular  opening  in  the  skull.  The  device,  shown  in  Figure  2, 
will  be  mounted  in  the  circular  opening  in  the  skull  and  coupled  to  an  external  connector  by 
a  thin  cable.  The  chip  and  associated  mounting  media  will  then  be  bonded  to  the  skull  using 
a  polymalient  glass  ionomer  cement.  Three  stainless  steel  bone  screws  will  be  tapered  into 
the  skull  at  approximately  equal  distances  around  the  circumference  of  the  trephine  opening 
for  use  in  anchoring  the  external  connector.  Liquid  dental  acrylic  will  then  be  poured  over 
the  exposed  surface  of  the  skull  and  the  anchoring  screws  forming  a  skull  cap  which  will  be 
used  to  provide  support  for  the  external  connector. 


Figure  2.  Mounted  AFIT  Array. 


Once  the  dental  acrylic  has  hardened,  the  skin  and  subcutaneous  tissues  will  be  apposed 
to  the  surface  of  the  skull  cap  using  a  pruse  string  suture  pattern.  The  animal  will  then  be 
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allowed  to  recover  from  anesthesia.  An  animal  caretaker  will  monitor  the  animal  continuously 
post  surgically  until  it  is  able  to  maintain  itself  in  an  upright  sitting  position. 

Seven  days  following  surgery  EEG  data  and  Visual  Evoked  Response  (VER)  data  will  be 
collected  from  the  animal  in  it’s  normal  awake  state.  This  process  will  require  the  animals  to 
be  restrained  for  approximately  10  minutes  each  day  during  the  recording  period.  The  ferrets 
will  be  restrained  by  simply  having  a  technician  hold  the  animal  on  their  lap.  The  non- human 
primate  will  be  restrained  by  placing  the  animal  in  a  commercially  available  restraint  chair 
which  has  been  specifically  designed  for  this  species.  The  monkey  will  be  adapted  to  the  chair 
during  three  weeks  of  training  conducted  prior  to  surgery.  It  is  essential  that  the  animals  be 
comfortable  and  free  of  distress  during  the  recording  period  not  only  for  humane  reasons  but 
also  to  assure  the  validity  and  reproducibility  of  the  resulting  data. 

The  animal  will  be  monitored  as  long  as  the  chip  is  functioning.  At  the  time  the  chip 
becomes  non-operable  it  will  be  removed  using  identical  surgical  procedures  as  were  employed 
during  installation.  A  stainless  steel  plate  will  then  be  placed  over  the  defect  in  the  skull  and 
secured  in  place  using  bone  screws.  The  skin  incision  will  be  closed  using  subcutaneous 
sutures. 

b.  Statistics:  This  study  will  involve  only  one  animal,  therefore,  no  statistical  tests  will 
be  performed. 

9.  Literature  Review: 

A  literature  search  was  conducted  using  Medline,  Agricola,  and  Biological  abstracts  data 
bases  to  assure  this  study  was  not  a  duplication  of  one  already  reported  in  the  literature. 
Based  on  the  results  of  this  search  it  is  determined  that  this  study  is  unique.  Likewise,  a 
DTIC  search  (search  #DKJ35A)  conducted  on  18  April  1994  revealed  no  indication  that  this 
work  has  been  accomplished  previously. 

10.  Animal  Utilization: 

Species:  Macaca  mulatta  Species:  Mustela  putorius 

Sex:  Male  Sex:  Male 

Weight:  12  KG  Weight:  1  KG 

Age:  12  to  15  years  Age:  12  months 

Total  number  required:  1  Total  number  required:  3 

a.  Three  ferrets  are  requested  in  phase  one  of  the  study  in  the  event  that  initial  testing 
reveals  the  chip  requires  minor  modification  and/or  additional  evaluation  prior  to  its  implant¬ 
ation  into  a  non-human  primate.  If  the  chip  appears  to  be  functioning  as  expected  following 
implantation  into  the  first  or  second  animal,  phase  one  testing  will  be  halted. 

b.  Use  of  Alternatives:  The  use  of  alternatives  such  as  computer  simulations  or  in  vitro 
techniques  were  considered.  However,  the  intent  of  this  research  is  to  study  the  neural 
electrical  activity  of  the  cerebral  cortex  as  it  relates  to  the  processing  of  visual  images  within 
the  mammalian  brain.  No  in  vitro  or  mechanical  system  yet  developed  can  approach  the  level 
of  sophistication  required  for  this  study. 

c.  Rationale:  Ferrets  were  selected  for  phase-one  of  the  study  in  an  attempt  to  use  the 
lowest  phylogenetic  species  of  animal  possible  to  obtain  preliminary  data  on  the  operational 
capabilities  of  the  chip.  Primary  requirements  were  that  the  animal  must  have  a  well  defined 
visual  cortex  and  a  brain  of  sufficient  size  to  mount  the  chip  and  associated  connecting  devie. 
Prior  research  by  other  authors  supports  the  use  of  ferrets  in  visual  system  research  (Jackson 
and  Hickey,  1985;  Wen  and  Shek,  1985). 

d.  Primate  Justification: 
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(1)  There  are  four  primary  reasons  the  non  human  primate  was  selected  for  this  study. 
First,  the  study  requires  a  species  with  a  brain  in  which  the  visual  cortex  is  surgically 
accessible  and  of  sufficient  size  to  allow  placement  of  a  0.5  X  0.5  cm  chip.  Second,  later 
stages  of  this  research  will  require  a  species  which  can  be  readily  trained  to  respond  to 
visual  images  originating  either  through  the  normal  optic  pathways  or  via  electrical  impulses 
delivered  directly  to  the  visual  cortex.  Third,  use  of  the  non  human  primate  allows  more 
efficient  use  of  data  already  collected  in  the  highly  sophisticated  human  visual  system.  Fourth, 
data  obtained  from  this  study  will  be  directly  applicable  to  the  human  species. 

(2)  The  study  is  not  designed  to  require  euthanasia  of  the  animal. 

e.  Post  Experimental  Disposition:  Ferrets  will  be  added  to  an  existing  ferret  colony 
within  the  facility.  Likewise,  the  non-human  primate  will  be  returned  to  the  primate  colony 
following  surgical  removal  of  the  chip.  In  the  event  the  health  of  the  animal  deteriorates 
or  for  some  other  reason  euthanasia  is  indicated,  sodium  pentobarbital  will  be  administered 
intravenously  to  the  animal  in  an  amount  sufficient  to  elicit  a  painless  death.  If  euthanasia 
is  performed  for  the  non-human  primate  due  to  poor  health,  it  is  felt  that  the  body  parts 
would  not  be  useful.  On  the  other  hand,  if  euthanasia  is  prescribed  for  reasons  other  than 
deteriorating  health,  an  attempt  will  be  made  to  distribute  organs  and  other  body  parts  to 
other  investigators  within  the  laboratory  who  may  have  a  requirement  for  these  tissues. 

11.  Relief  of  Pain  or  Distress: 

a.  This  study  is  classified  as  category  B  in  the  Wright-Patterson  AFB  “Pain  and  Distress 
Classification  System.”  “The  research  potentially  involves  minor  short-term  pain,  discom¬ 
fort  or  distress  which  will  be  treated  with  appropriate  anesthetics/ analgesics.”  The  surgical 
procedures  involved  in  this  study  would  cause  pain  to  the  subject  animal  if  not  relieved 
with  anesthetic  drugs.  Human  patients  who  have  had  similar  devices  installed  within  their 
skulls  to  record  and  provide  electrical  stimulation  to  their  visual  cortex  have  not  indicated 
the  procedure  was  painful. 

(1)  The  animal  will  be  pre  anesthetized  with  Ketamine  HCL  (15  mg/kg)  and  atrophine 
(0.04  mg/kg).  Surgical  anesthesia  will  be  provided  using  a  3-5%  concentration  of  Isoflurane 
superimposed  on  a  80/20  mixture  of  oxygen  and  nitrous  oxide.  Post-operative  analgesia 
will  be  provide  by  administering  Buprenorphine  (0.01  mg/kg)  twice  daily  by  intramuscular 
injection  for  4  days  or  longer  if  deemed  necessary. 

(2)  NA 

(3)  The  following  data  bases  were  searched  in  an  attempt  to  determine  a  less  painful  or 
distressful  means  of  conducting  this  study:  Agricola,  Biological  abstracts  and  Medline.  No 
less  painful  or  distressful  means  of  conducting  the  study  were  found. 

(4)  Death  is  not  an  endpoint  in  this  study.  Should  the  animal  become  debilitated,  loose 
in  excess  of  10%  of  its  body  weight,  refuse  to  eat  or  drink,  or  in  some  other  way  indicate 
that  it  was  in  non  relievable  pain,  it  will  be  humanely  euthanized. 

(5)  Dr.  John  Latandresse  DVM/PhD  was  consulted  in  the  design  of  this  study. 

12.  Personnel  Training: 

The  surgical  procedure  will  be  performed  by  a  veterinarian  who  has  had  previous  ex¬ 
perience  in  the  placement  and  removal  of  cortical  electrodes  and  skull  caps.  All  animal 
manipulations,  and  post  operative  care  procedures  will  be  performed  by  personnel  certified 
by  the  American  Association  for  Laboratory  Animal  Science  (AALAS)  at  the  technologist 
level. 
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Appendix  C:  Post-processing  steps 


C.l  Standard  Clean 

1.  Mechanically  agitate  chips  in  an  acetone  bath  for  30  seconds. 

2.  Mechanically  agitate  chips  in  a  methanol  or  propanol  bath  for  1  minute. 

3.  Bake  chips  at  90°  C  for  15  minutes. 

NOTE:  Do  not  use  ultrasound  cleaner  with  chips.  It  does  visually  undetectable  damage  to 
the  chips. 

C.2  Metalization  Process 

1.  Clean  with  standard  clean  process. 

2.  Remove  from  oven  and  allow  to  cool. 

3.  Apply  adhesion  promoter  (HMDS) 

-  Puddle  HMDS  onto  the  chip.  Allow  it  to  spread  over  the  entire  chip  and  coat  the 

edges. 

-  Allow  to  sit  for  5  seconds  to  ensure  good  coverage  of  the  chip. 

-  Spin  at  3000  RPM  for  45  seconds. 

4.  Apply  positive  photo-resist  (Shipley  AZ1350J) 

-  Puddle  AZ1350J  onto  the  chip.  Allow  it  to  spread  over  the  entire  chip  and  coat  the 

edges. 

-  Allow  to  sit  for  5  seconds  to  ensure  good  coverage  of  the  chip. 

-  Spin  at  4000  RPM  for  45  seconds. 

5.  Place  chip  at  the  edge  of  Pyrex  container  so  that  only  the  only  one  edge  of  the  chip  touches 

the  bottom,  and  two  corners  touch  the  side  of  the  container. 

NOTE:  Chips  that  are  laid  flat  on  the  Pyrex  dish  will  become  fastened  to  the  bottom,  and 
must  be  cleaned  with  the  standard  clean  process  before  being  removed.  Otherwise,  the 
chips  will  break. 
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6.  Pre-bake  at  90°  C  for  20  minutes. 


7.  Align/Expose  for  1.1  minutes. 

8.  Immerse  in  chlorobenzene  for  10  minutes. 

9.  Bake  at  90°  C  for  15  minutes. 

10.  Develop  photo-resist. 

-  Spin  at  500  RPM 

-  Spray  with  AZ351:DIW  1:3  for  45  seconds. 

-  Spray  with  DIW  for  30  seconds. 

-  Spin  dry  at  5000  RPM  for  45  seconds. 

11.  Examine  pattern.  If  further  develop  is  necessary,  develop  for  an  addition  time. 

12.  Post-Bake  at  90°  C  for  15  minutes. 

13.  Sputter  Ti  for  25  minutes  with  forward  power  of  200  W  (Approx.  300  microns). 

14.  Sputter  Ir  for  75  minutes  with  forward  power  of  150  W  (Approx.  3000  microns). 

15.  Immerse  in  acetone  bath.  Lightly  scrub  with  a  cotton  swab  to  ensure  all  undesired  metal 

is  lifted  off  of  the  chip. 

C.3  Polyimide  Application  Process 

1.  Clean  chips  with  the  standard  clean  process. 

2.  Remove  from  oven  and  allow  to  cool. 

3.  Apply  polyimide  (PI-2722) : 

-  Puddle  PI-2722  onto  the  chip  so  that  it  flows  over  all  edges. 

-  Allow  to  sit  for  5  seconds  to  ensure  good  coverage  of  the  chip. 

-  Spin  at  3000  RPM  for  30  second  (Approx.  10pm). 

4.  Soft-bake  at  55°  C  for  90  minutes.  -  Dries  the  polyimide  without  curing. 
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5.  Remove  from  oven  and  allow  to  cool. 

6.  Align/Expose  for  4.5  minutes. 

7.  Etch  the  polyimide. 

-  Spin  at  500  RPM 

-  Spray  with  AZ351:DIW  (1:3)  for  20  seconds. 

-  Overlap  spraying  with  DIW  for  7  seconds. 

-  Spray  with  DIW  for  20  seconds. 

-  Spin  dry  at  5000  RPM  for  30  seconds. 

8.  Examine  to  ensure  the  etch  is  complete.  Etch  for  additional  time  if  it  is  required. 

9.  Final  cure  at  300°  C  for  30  minutes.  Complete  cure  of  the  polyimide. 

CA  Packaging 

1.  Mount  the  chip  on  a  header  package  and  wire-bond  the  connections. 

2.  Apply  a  coat  of  polyimide  as  follows: 

-  Allow  polyimide  to  reach  room  temperature  prior  to  opening. 

-  Using  a  toothpick,  apply  polyimide  over  the  wire-bonded  connections  being  careful 

not  to  cover  any  of  the  electrode  array  or  reference  electrode. 

3.  To  avoid  expansion  and  contraction  of  the  polyimide  layer  which  could  break  wire  bonds, 

the  polyimide  is  cured  using  gradual  changes  of  temperature  as  listed  in  Table  C.4. 

4.  Allow  chips  to  cool  in  oven. 
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Temperature  (°  C) 

Time  (min.) 

70 

30 

85 

30 

105 

30 

130 

30 

165 

360 

Table  C.l:  Cure  times  for  final  application  of  polyimide. 
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Appendix  D:  Training  Pulse  Coupled  Neural  Networks  (PCNN) 

The  pulse  coupled  neural  network  is  a  detailed  model  of  a  biological  neuron  that  exhibits 
the  ability  to  encode  both  spatial  and  temporal  signals  into  a  temporal  map  [6].  The  typical 
PCNN  model  has  three  parts:  the  dendritic  tree,  linking  dendrites  and  the  pulse  generator 
[6,  2],  The  dendritic  tree  is  the  main  pathway  for  information  entering  the  system,  where 
the  feeding  input  dominates  the  internal  activity.  The  linking  portions  of  the  dendrites  allow 
interaction  between  neurons,  allowing  synchronization  of  firings  and  modulation  of  the  feeding 
input.  And  the  pulse  generator  passes  information  out  of  the  neuronal  model  as  a  synaptic 
firing.  All  synapses  in  this  model  are  viewed  as  leaky  integrators  [6,  2].  If  a  particular 
neuron’s  internal  activation  has  risen  near  it’s  threshold  activation  level  for  firing,  a  linking 
neuron  can  fire  and  push  it  over  the  threshold,  making  it  fire  synchronously  with  the  linking 
neuron.  Alternatively,  the  linking  neuron  could  fire  and  inhibit  the  subject  neuron  from  firing. 

The  PCNN,  figure  D.l,  is  a  basic  building  block  (advanced  perceptron)  for  higher  order 
networks.  The  linking  fields  allow  an  endless  number  of  possible  organizations  and  subsequent 
logical  operations,  because  linking  can  inhibit  or  excite  [6] .  Additionally,  pulse  coupling  adds 
a  dimension  of  time  to  an  MLP.  Two  lower  level  neurons  will  have  much  less  effect  on  an 
upper  level  neuron,  than  if  they  were  synchronized.  This  is  due  to  the  time  decay  of  the 
internal  activation. 

D.0.1  Basic  Design. 

The  output,  Yj(t),  of  a  pulse-coupled  neural  network,  using  Johnson’s  design  is  given  by 


Yj (f)  =  step  (Uj (t)  -  9j (i))  (D.l) 

[6].  Because  the  internal  activation,  Uj(t)  increases  at  a  finite  rate,  while  the  threshold,  6(t) 
increases  with  an  infinite  slope  at  each  firing,  Yj(t)  is  an  impulse.  When  Uj(t )  >  6j(t)  then 
the  neuron  fires,  raising  6j  (t)  above  Uj(t)  instantaneously.  For  discrete  time  steps  it  is  easier 
to  use 


Yj(t)  =  { 


(D.2) 


1  if  Uj(t)  >  0j  (t) 

0  otherwise 

producing  a  pulse,  or  firing,  output  when  the  total  internal  activity,  Uj  (t) ,  rises  above  the 
threshold,  0j(t)  [2]. 
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Figure  D.l:  Pulse  Coupled  Neural  Network  [6] 


The  internal  activation,  Uj(t),  is  a  function  of  a  feeding  input,  Fj(t),  and  any  number  of 
linking  inputs,  Lij(t).  The  linking  inputs  are  weighted  by  the  linking  coefficient,  /?*■  j . 


Uj(t)=F3(t)H(l  +  f3ijLij(t))  (D.3) 

i 

The  feeding  input,  Fj(t),  is  a  function  of  an  optional  analog  input,  Ij(t ),  and  weighted, 
summed  synaptic  inputs  from  other  neurons,  Yj-f  ( t ).  Sensory  neurons  usually  do  not  include 
inputs  from  other  neurons,  while  higher  level  neurons  do  not  typically  have  an  analog  input, 
Ij.  Each  synapse  between  the  k—  feeding  neuron  and  the  j—  neuron  is  modeled  as  a  leaky 
integrator  with  synaptic  weight  Mkj  and  decay  time  constant  a Therefore, 

Fj{t)  =  m  +  Y,F^)  (D.4) 

k 

where,  using  V  as  the  convolution  operator 

Fkj(t)  =  (Mkje-01^)  *  YkF(t )  (D.5) 

[6].  Using  a  digital  filter  representation 

Fkj  (t)  =  Fkj ( t  -  At)  ■  e~< • A<  +  MkjYk(t)  (D.6) 


[2]. 
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The  i—  linking  dendrite  activity,  L2j(£),  is  a  function  of  weighted,  summed  synaptic  inputs 
from  other  neurons,  Yk(t).  Each  synapse  between  the  A—  linking  neuron  and  the  i—  linking 
dendritic  branch  of  the  j—  neuron  is  modeled  as  a  leaky  integrator  with  synaptic  weight  Wkij 
and  decay  time  constant  a^- .  Therefore, 

Lij(t)  =  Y,Lkij(t)  (D.7) 

k 

where 

Lkij  ( t )  =  (Wwie-<t)  *  YkL(t)  (D.8) 

using  a  digital  filter  representation 

Lkij  (t)  =  Lkij  (t  -  At)  •  e~<'At  +  WkijYkL(t)  (D.9) 

[6,  2]. 

The  threshold,  8j(t ),  is  a  function  of  the  output,  lj,  and  is  also  modeled  as  a  leaky 
integrator  with  amplitude  gain  Vf  and  time  constant  a®.  0jo  is  added  as  an  offset.  Oj(t)  gets 
charged  every  time  the  neuron  fires.  This  keeps  the  threshold  above  the  internal  activity,  so 
that  the  neuron  isn’t  in  a  continuous  firing  state.  The  threshold  exponentially  decays  to  allow 
firing  to  continue  after  a  delay. 

Oj  (0  =  ejo  +  (vfe~aP)  *  Yj(t)  (D.10) 

using  a  digital  filter  representation 

**(*)  =  0JO  +  6j(t  -  At)  •  e-<A<  +  VfYj(t)  (D.ll) 

[6,  2]. 

D.0.2  Learning  Laws .  Most  of  the  literature  on  PCNNs  deals  with  methods  for  optical 
systems  which  perform  image  processing,  however,  the  PCNN  can  be  a  powerful  pattern 
recognition  tool.  With  standard  image  processing  weights  are  set  by  a  person  to  perform  a 
desired  task;  segmentation  or  smoothing.  To  be  useful  in  classification,  the  PCNN’s  weights 
must  be  adapted  based  on  training  samples. 

The  PCNN  can  be  trained  in  either  a  batch  or  an  instantaneous  mode.  In  the  batch  mode, 
the  PCNN  becomes  an  advanced  perceptron  or  MLP.  The  instantaneous  mode  is  not  quite 
as  common  in  artificial  neural  networks,  but  is  very  straight  forward. 
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Additionally,  a  combined  learning  mode  is  possible,  that  can  capitalize  on  the  benefits  of 
both  previously  described  modes. 

There  are  two  ways  to  view  synchronization  for  training  purposes.  First,  because  syn¬ 
chronized  neurons  have  a  stronger  effect  on  the  network  than  unsynchronized,  therefore  train¬ 
ing  linking  weights  in  the  same  batch  mode  as  other  weights  will  capitalize  on  synchronization. 
Second,  if  synchronization  is  viewed  as  an  end  to  itself,  then  linking  weights  must  be  updated 
to  cause  the  desired  level  of  synchronization. 

D.  0.2.1  Batch  Mode:  PCNNs  as  Perceptrons.  Both  PCNNs  and  Perceptrons  take 
an  input  and  compute  a  firing  rate  based  on  the  input  and  synaptic  weights.  The  PCNN  does 
perform  a  more  precise  encoding  of  the  input  than  the  perceptron  because  of  its  temporal 
firing  sequences,  but  all  of  the  linking  could  easily  be  implemented  in  an  MLP  by  adding 
feedback  loops. 

To  allow  the  use  of  perceptron  type  learning  for  a  PCNN,  a  time  ‘batch’  must  be  defined. 
Therefore,  let  T  =  n(At)  where  n  {Naturals}.  Now  the  average  firing  rate,  i>j{b)  for  the 
period  P  can  be  computed. 


$j(b)  =  Oil p  ±  wAt))  (D.12) 

where  b  represents  the  batch  process,  making  / j  the  first  sample  of  batch  b. 

Training  the  PCNN  to  reduce  the  total  error  requires  the  following  definition  of  error. 

E(b)  =  l't(d*-Mb))‘2  (D.13) 

Z  z= 1 

where  J  is  the  total  number  of  ‘output’  neurons  of  interest.  These  neurons  do  not  have  to 
be  output  neurons,  but  they  must  be  neurons  for  which  a  desired  average  firing  rate  can  be 
asserted.  Now,  all  synaptic  and  linking  weights  of  the  output  neurons  can  be  updated  in  the 
direction  of  the  negative  slope  of  the  error  surface,  with  an  arbitrary  step  size,  rj. 


W(b  +  1)  =  W(b)  -  p 


dE(b ) 
dW 


where  ‘IT’  represents  a  generic  weight,  synaptic  or  linking. 


(D.14) 
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An  algorithmic  simplification  can  be  made  for  multi-layered  and/or  multi-connected  net¬ 
works,  by  removing  the  requirement  for  a  predetermined  desired  average  firing  rate.  This 
simplification  would  be  to 

1.  Apply  the  error  equation  to  the  output  neurons  based  on  real  desired  firing  rates 

2.  Update  weights  for  output  neurons  (including  the  linking  weights  and  possibly  even  the 
decay  time  coefficients).  Keep  a  record  of  the  direction  and  magnitude  of  each  change 
to  synaptic  weights.  These  weight  changes  will  be  passed  to  the  input  neurons.  If  the 
jtJk  neuron  increases  its  synaptic  weight  for  the  ktJ±  input  neuron  by  0.1,  then  the  k^ 
input  neuron  is  requested  to  increase  its  firing  rate  by  0.1  (or  a  scaled  amount). 

3.  For  each  lower  level,  or  input  neuron,  total  the  changes  recorded  in  step  2,  associated 
with  the  subject  neuron.  This  will  give  a  relative  direction  and  amount  of  change 
required  in  the  firing  rate  (4+5  means  rate  needs  to  increase  4— 9  means  rate  needs  to 
decrease) .  This  can  be  scaled  and  used  as  the  desired  for  the  lower  level. 

4.  Make  sure  you  did  not  update  weights  on  a  neuron  that  has  already  been  updated  in 
this  batch,  to  avoid  infinite  loops. 

To  update  flij  (from  equation  D.14): 


fcJ(b+l)=(3iJ(b)-v?^- 

from  equation  D.13 


(D.15) 


dE(b) 

dflij 


d 

dfaj 


L  Z- 1 


d  r  i 

dfiij  [2  I  ~  “l - f  (dj  ~  V\/(&))2} 

m  [s (di  ~  *(6))1 


(di  -*m  (-vjjj-wm 


Looking  back  to  equation  D.12 


(D.16) 

(D.17) 

(D.18) 

(D.19) 


dfiij 


d 

dfi, 

1 


n-i 


EwZo  (Yj(tb  +  wAt)) 

r 


FSteK('*  +  "A,)Ij 


(D.20) 

(D.21) 


From  equation  D.l,  and  letting  t  =  tf,  +  wAt: 
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WjYj^  =  Wij  fstep^^r^  ~  9j^  (D-22) 

At  this  point,  the  step  function  makes  direct  differentiation  impossible.  Close  inspection, 
however,  shows  that  the  only  purpose  for  continuing  the  differentiation  is  to  decide  whether 
to  increase  or  decrease  the  weight  in  question,  or,  more  specifically,  what  effect  a  change  in  the 
particular  weight  will  have  on  the  output.  The  term  ‘( dj  -  ^j(b))  (—1)’,  from  equation  D.19, 
already  determines  whether  the  firing  rate  needs  to  be  increased  or  decreased,  and  by  how 
much. 

To  continue  differentiation,  one  must  capitalize  on  the  nature  of  the  pulse  generator, 
equation  D.l.  Firing  occurs  when  Uj(t)  rises  above  9j(t ).  Therefore,  increasing  Uj(t)  will 
have  the  effect  of  increasing  the  firing  rate.  The  remaining  term,  6j(t),  is  solely  a  function 
of  the  output  (reference  equation  D.10),  and  will  therefore  not  be  effected  by  any  change 
in  the  weights.  Because  Yj(t)  relies  completely  on  Uj(t)  in  a  direct  fashion,  the  following 
substitution  will  only  lose  a  scaling  factor,  which  can  be  accounted  for  in  the  step  size. 


d 

dp, 


Uj(r)  = 


d 

dp, 


Fj(T)  nu+ PzjLzjir)) 


z= 1 


d 


Uj{T)  JJ  (1  +  PzjLzj{r ))  Qp  [1  +  PijLij(T )] 


z^i 

Uj{r) 


(1  +  PijLij(r)) 


Uj(T) 


From  equation  D.21 


di>j{b)  i  ^  [  Uj  T  .  , 

dPij  ~  r  Y  [ (1  +  Pii(b)Lij{T))  lJ (r) 


Substituting  into  equations  D.15  and  D.19 


Pij(b  +  1)  —  {b)  +  p  {dj  -  'ipjib))  Y 


n-i 


ty=0  L 


Uj 


(1  +  Pij(b)Lij(T)) 


Lij{r) 


To  update  Wkij,  equation  D.25  can  be  a  starting  point. 


(D.23) 


(D.24) 

(D.25) 

(D.26) 


(D.27) 


(D.28) 
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m = F> m  n  a + /w*  w)  n + 


(D.29) 
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<9^u 


(1  +  PijIJii{T))  —  flij  7^777  Lij(r) 


dWh 
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Pa 
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tJ  dwkijLkij{T) 


P 


d[Lkij(r)e-a^Ar  +  WkijYkL(r)} 


y 3 


dWk 


IJ 


=  PijYkL(r) 


(D.30) 

(D.31) 

(D.32) 

(D.33) 

(D.34) 
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Wij(b  +  1)  =  Wij(b)  +  ±  (dj  -  ^(b))  E 
To  update  Mkj,  equation  D.24 


w=0 


Uj 


(1  +  Pij{b)Lij(r)) 


ykL(r) 


using  equation  D.4 
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()Mh 
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(D.35) 


(D.36) 

(D.37) 

(D.38) 

(D.39) 

(D.40) 


Mkj(b+  1)  =  Mkj(b)  +  1  (dj  -  Ub))  E 

iu=0  L 


n-i 


Uj  VF 


Fj(b) 


y£  (r) 


(D.41) 


Using  the  same  derivation  as  above,  a  learning  rule  can  be  established  for  all  time  con¬ 
stants. 

D.  0.2.2  Instantaneous  Mode.  Instantaneous  mode  training  can  be  viewed  in  a 
number  of  ways,  all  retaining  the  goal  of  training  the  PCNN  at  each  time  increment  and 
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based  on  the  information  at  hand.  The  particular  method  used  herein  considers  a  synapse  as 
the  driving  force  for  weight  change. 

Competing  hypertrophic  and  atrophic  forces,  where  hypertrophy  is  caused  when  a  synapse 
occurs  (increasing  the  weight  given  to  that  synapse)  and  atrophy  is  a  constant  exponential 
decay  of  the  weight,  gives  a  self  organizing  character  to  the  PCNN.  Because,  with  even  a 
limited  amount  of  linking,  this  training  process  may  saturate  the  output  of  numerous  neurons 
by  increasing  synaptic  and  linking  weights  of  symbiotic  neurons  to  unacceptable  levels,  the 
learning  law  must  constrain  the  weights.  With  this  in  mind,  consider  the  following  learning 
law. 


Wt=(  1-—)  Wt- At  +  —  (1  -  Wt-  At)  rkt_At  (D.42) 

where  Wt  and  Wt-At  are  the  new  and  previous  weights,  respectively,  to  be  substituted  with 
Mkjt  Wkij  and  flij  from  equations  D.6  and  D.9.  Also  aa  and  are  the  atrophic  and 
hypertrophic  factors,  respectively,  where  aa  >  a^.  This  rule  keeps  all  weights  between  0  and 
1. 

Training  in  this  mode  requires  less  memory  and  CPU  time  compared  to  batch  mode,  but 
reduces  the  possible  complexity  of  the  neural  interaction  (whether  a  linking  input  excites  or 
inhibits  must  be  predetermined). 

D.  0.2.3  Combined  Mode.  This  mode  uses  the  fast  and  simple  weight  update  rule  for 
Mkj  and  Wkij’,  but  implements  the  batch  update  ride  for  / 3*j .  This  allows  the  linking  weights 
to  inhibit  or  excite  based  on  the  training  samples  and  the  desired  output,  while  maintaining 
the  speed  of  the  instantaneous  mode. 
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